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ABS TRACT 

In order to reduce school shop accidents and foster 
safe working habits, the industrial arts teacher must have a 
thorough working knowledge of the equipment and/or processes 
he is demonstrating and subsequently having his students 
practice. 

In order to develop this thorough working knowledge and 
foster working habits the student teacher must in his labora- 
tory training be made aware of the many contingencies both 
he and his students will encounter in their various activi- 
ties. The student teacher must develop an awareness of the 
knowledge he has to possess in order to function at an optimum 
level in the industrial arts laboratory. There is also need 
to develop in those same student teachers an awareness of 
the responsibility they will face to conduct their classes 
in as safe an environment as is possible and at the same 
time instill in their own students that same awareness of 
safety that will continue from the school laboratory into 
their lives beyond the school. 

The purpose of this study was to develop and explore 
the feasibility of a computer managed instruction system to 
deliver a series of safety tests: tests that would provide 
the instructor with an indication of student strengths, give 
the student specific remediation when required and provide 
for that student an inventory of learnings to maintain an 


awareness of personal achievement. 
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The system was also designed to try to develop a safety 
knowledge that would foster safe working habits and develop 
safety awareness and responsibility that would be carried 
from the university laboratory into the school shop. 

The procedures used for this study were divided into 
five stages. Stage 1, the development of the test material 
which concentrated on the development of specific test 
objectives, test topics and the writing of suitable test 
items. Stage 2 saw a review of the safety tests to make 
sure they conformed to the objectives, and in stage 3 the 
safety tests were authored into the PLATO Learning Management 
System. In stage 4 an analysis of results was carried out 
and finally stage 5 concentrated on an assessment of the 
Suitability of the PLATO Learning Management System to deliver 
and manage the safety tests. 

The research indicated that the use of the computer 
managed instruction in safety education was viable in that 
it provided for the instructor sufficient feedback regarding 
student progress, strengths and difficulties. Secondly the 
research indicated that the system was able to provide 
feedback to the student regarding personal achievement. The 
third) anc tinal part of this’ research, that of an analysis of 
results, was unable to be carried out completely as a number 
of data set translation problems arose that were unable to 
be resolved 2t the, time Of Writing. The partial analysis 
that was undertaken did not indicate that the PLATO 


Learning Management System was, through its data gathering 
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capability able to provide an indication vOL Overly GlhELlcuLe, 


overly easy or ambiguous test items. 
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"Life is intrinsically dangerous. 
Life is partly routine, to be sure, but 
more fundamentally it is an experience of 
the unknown and hence based on adventure 
... The prime quality in safety, therefore, 
is not the removal of danger but an improve- 
ment in the quality of adventure" 


(Whitney, A.W., Safety for Greater 
Acventure, an Stack, Hive, and 
J.D. Elkow, Education for Safe 
Having, 1966, .p.13) 
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CHAPTER 1 
INTRODUCTION 


ORIENTATION TO THE PROBLEM 


The human element is the most significant factor related 
to accident causation. The correction of such human failing 
is dependent upon a number of factors, Frenze (1973) notes: 

"One of the principal reasons why unsafe 

acts are committed... is that those who commit 

them don't know what they do is unsafe or 

ENCOPrect i ato. 33) 

Frenze further notes that the correction of the unsafe 
act depends upon training, education and motivation to ensure 
procedures are followed. (ibid) 

In a school laboratory environment the student is faced 
with a series of decisions regarding the task he is to per- 
form and the machine or process he is to use to complete the 
task. In order for the student to move towards his objective 
a number of processes have to take place. First the student 
must make decisions and based on those decisions take certain 
risks. To do this, the student first needs information upon 
which to base his decisions; the better the information the 
better the decision, and subsequently the more calculable 
thietrisk anal gathe tstudent is.4information ¢us¢poor.or insuffi-= 
cient there is greater chance for a poor decision being made 
and hence a greater risk. Secondly, if the student's task 
involves the use of a machine, that machine must function 


without failure: poorly maintained machines or those misused 


due to lack of safety knowledge could themselves be the cause 
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of an accident. The reduction and/or elimation of school 
shop accidents then becomes dependent in large measure on 

the ability of the teacher to anticipate unsafe acts and 
recognize the steps that should be taken to ensure proper 
behavior. A superficial knowledge of operational procedures 
is not enough to ensure safety; therefore the teacher must 
become familiar with each piece of equipment, its parts, the 
correct handling procedure, the hazards involved and the pre- 
cautions necessary to guard against those hazards, (Florio 
andestaftford, 19695» p.303) 

The development of knowledge and its subsequent evaluation 
in the area of safe operational procedures then becomes a 
very important factor in the training of industrial arts 
teachers. 

This study tested the feasibility of a computer managed 
instruction (CMI) system, specifically the PLATO Learning 
Management System (PLM) to deliver a series of safety tests 
for various pieces of equipment and processes used in the 
industrial arts metalworking laboratories at the University 
of Alberta. The safety tests were directed toward those 
prospective industrial arts teachers enrolled in EDIND 211 
(Metals and Materials 1) and EDIND 412 (Metals and Materials 
lAhetofprovade, \for the linstnuctonjva .systematicrcheckaof 
student readiness to perform in a safe manner. 

The use of the computer managed instruction system was 
based on its perceived role in education and its potential 


for use in the delivery of the safety tests. Baker, F.B. 
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(1978), notes that with the use of CMI the teacher is freed 
from many clerical-level tasks through the automation of 
diagnosis and prescription at the unit-of-instruction level, 
which allows more time to be put to instructional-related 
tasks. Baker also notes a shift by the teacher towards a 

more managerial role, one endorsed by Ziel (1971, p.95) 

in his description of the industrial arts teacher. With the 
use of CMI "the teacher-manager is now in a position to 

manage on a basis of data rather than intuition and judgement". 
(Baker, FSB. ,/ 19787) 01.1386)i,5 ani important consideration: in 


the area of safety. 


NEED: FOR THE STUDY 

In 1976, the last year for which figures are available, 
there were more than 1,071,000 work related injuries and 
illnesses among a total of 8,500,000 workers employed in 
Canada. (Workers Compensation Board, cited Clark, 1979, p.2) 
These figures do not include those injuries not claimed or 
not covered by the Workers Compensation Board. Clark also 
notes that approximately 54 percent of these injuries and 
illnesses involved no loss of time except on the day of the 
injury, however the remaining 46 percent accounted for the 
loss vofvalmost) dz mil lionwork days? eachtinjury Ganvolving 
a Hoss) Of approximately) 25 workidays.® [hws dloss! 1s testimated 
Go ibexequlvalenteto the Woss: ofiproductiion: for ioneitulimyear 
of 49,000 full-time workers. Over the nine year period 


1969-1978 work related injuries and illnesses increased 35 
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4 
percent overall, disabling injuries increased 69 percent and 
non-disabling injuries 15 percent. It should be noted how- 
ever that in this nine year period employment increased 29 
percent causing the accident incidence rate (the number of 
injuries and illnesses per one hundred workers) to increase 
onlyimarginallysetromel2slatosl2. 6) oThissfigure sis thowever 
disputed by Clark (1979) who suggests that it could be as 
much as 20 percent higher due to statistical error. (p.2) 
The cost in medical benefits of these work-related injuries 
WenE Up L£romes2/4emilivoneinel969 torcS9726¢mbllion innl978:. 
In terms of total industrial cost however the figure was in 
fact a great deal higher, using the ratio developed by Heinrich 
(1959) of 4:1.4 “thestobal loss aconcept))) that¢is, indirect 
costs, which include pecuniary-measures for failure of busi- 
ness to meet customer demand, lowered employee morale, 
decreased efficiency and lost supervisory time, to direct 
costs such as medical aid and compensation for lost earnings, 
the industrial costs amounted to $3.5 billion. (Work Injury 
Experience and Costs in Canadian Industry. (1969-1978) 

In the Province of Alberta, the number of work-related 
accidents and illnesses in 1973 (the first year of available 
figures) totalled 79,895 for a total working population of 
approximately 549,000. These accidents accounted for 1.9 
million work days lost at an incidence rate of 14.5 work 
days per one hundred workers. By 1977, the last year for 
which figures are available, the number had risen to 109,131 


work related injuries and accidents in a working population 


ae Paaciesg Pr beune 1s ned ; ea na if tok a ik ’ inci " ribet ) 
aw tna mae A 
tang, Sitter eat att ; 


a 


i. ‘ 
~ weet hados. ed hives Ae “9 
oe bender end semtselil 


6. faq sere sit} heed oui a 
ce 


cahane of (encod RBA Soe! #9] =saannhes fra age 
yapewbod- ei sult Skee) eet Sel sow ehlen iw 


aa 


es 9d bieo? a: 2am pics a omer tet et) sens wt 2 


a 1a 


(i, 4) «20006 fee teen sie or gui! ferip.ir rcraeraey os a 


Tes - av ‘peers Sh ah suo 


j 
>< 
y 
" 
haw. 


SU ee SRL. af 
; wrt bes 
re anv eiupil ai; teverenr Sacyp : SiAZ2 


res 
| ed 
+ > 
LP = 
a 
te 
bh 
o = 
P 


iniuoiet vd Beagbevah olf Ry See patel \tomeis [sek taerp & 
ney ihn Seb dale (4oecnaer meer Pade7 oft) . ich Fo, Gs 
bevd 30 etuibea ie pews pete Ytes deosy soiinnt fotten 
: e Bet | i ve ; 
ait st x beans Be AOS i Reet Yomecae Senne 
tastih (od @ennts, 4x erat ign bat yoauetotite hee 
apn toxns sagt tot a0 Lssamectil oe ‘ ing. Lec 2a - dont 
° - a ig x 
rubat ave) .netthon 268 a iene : 
‘ertreraas) es ah sate oa me 32202 bos sonetzoge 
_ x 
hetaioc~ Ancw 2c radii eid bavorir ne bon cos | wae, at _ 
i. au 
’ ‘eta vans is ban | 
- —— 
‘ Sane acai Ces 


’ 
v 
oT 
y we 4 


stow Rall iio met, lens c a 


-_ aaa ad ~ a 


eLdaliavs +5 1SBY, ¢ tens .2 oa 


' 39 a0 | folios pick tediw Lat0 


es. tout | Penienons & zane: 


5 

of 710,000. Days lost had increased to 2.7 million and the 
incidence rate to 15.3 work days per one hundred workers. 
(Alberta Workers Compensation Board Estimates of Employees 
by Province and Industry, nd.) 

The report of the Industrial Health and Safety Commission 
Of Alberta, 1976 (The Gale Commission) notes that the para- 
mount cause of accidents is the indifferent attitude of those 
involved (p.38). Furthermore, the commission also noted that 
attitudes towards safety are most easily developed in individ- 
uals in their formative years and once established in mature 
individuals are difficult to influence. (pp. 84-85) The 
commission report recommended therefore that: 

"the basic goal (of education) must be to 

raise the overall level of awareness towards 

all matters relating to ...safety" (p.96) 

The commission also recommended that the government play 
a vital leadership role through the Department of Worker's 
Health Safety and Compensation to co-ordinate all the educa- 
tional processes in this field: 

"It (the department) should support, guide 

and direct these (the educational processes) 

toward the ultimate goal, the worker at the work 

site and stimulate the various professional 

Organizations to establish standards of compe- 

tence...We also suggest that greater emphasis 

be placed on safety...education in the school 

Syerem (ped 7) 

During February 1979 a meeting on industrial education, 
Occupational health and safety chaired by Professor A.K. 


Deane of the Department of Industrial and Vocational Education 


at the University of Alberta was attended by industrial 


3 | SiG pai Btneb tone 46 secwe en 
panty! \Biadore sede: ae cee i, an tek ay Bae 
-hiviiad ai. Stiieighanst vt iia Be age YWoltte slrawos. seep 
oedam ae Roethe niet 739g oes te 

| attr (2ienn eee: ‘ aerees i. o3 ¢idott? Lb 226 etauvtiivll 
ated ote tirierarsoc 27 s2oge soles 
C (aod a Lede oistid ett” . ai 


od 30¢ aT to 4603! liviero ens eeiet’ 
oc erro x OF Saddeie: etoty sit abd 


¥eid snonereehebia ent teas baiiaiono.ss ®eis acl@aicomo oat © < 


* seul 4, teem ert. saigaeeeats Stor qitarebust tabs ‘s. 
~iatts Beto tit sis. eteodbrsoo 4 Caiteruscn bre worse’ : 


eee ality at eeeasseng 


pipet angel pda) oot: (4 + / 
ht) proee toesib bas © 

g Ateeisic sft brewed 9) 

Pa jie aalvalce bue, ef ia*~.. 

Sembee Gaver 08 vaoiseainepae, 0 
sa BID * ee Where orle nibita ae) 


eegeeiaen ao 
| a (rei mosey 


Rei settee 3 n itamnet yvisdy WE: a 


o the : ais ; im 7 x ce ms ae i 
ey ee vin ei shy Te 
‘vine d me ae 


education representatives from the Edmonton Separate School 
Board, the Provincial Department of Education and the 
Department of Occupational Health and Safety. The following 
areas of concern were identified: 

1 There is need for greater awareness of safety 
by teachers at all levels of education. 
Specifically, teachers should be made aware 
of the 1976 legislation on occupational health 
as it applies to the school situation. 

2 There is an urgent societal need for a change 
of attitude towards safety. 

and 

3 Schools need to meet minimum standards for 
safety as required by the 1976 provincial 
occupational health and safety legislation. 
(personal correspondence, A.K. Deane, 

February, 1981) 

A 1979 discussion paper emanating from the Research and 
Education Branch of the Occupational Health and Safety 
Division of the Alberta Department of Labour concludes that 
safety training in Alberta schools is generally inadequate. 

(personal correspondence, A.K. Deane, 


February, 1981) 
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SUMMARY 

An industrial arts teacher working in a student labora- 
tory environment is faced with potentially hazardous condi- 
tions that require knowledge of the many contingencies both 
he and his students will encounter in their various activi- 
ties. 

The preparation at the university level, of these same 
Industrial Arts teachers, in Metals and Materials requires 
therefore that their safe working knowledge of metalworking 
equipment and processes be assessed and evaluated and specific 
remediation be given when required. 

This process of knowledge assessment and remediation 
however, becomes especially difficult in the industrial arts 
laboratory because of the many and varied routes chosen by 
the student to complete the metalworking course. In the 
laboratory area the instructor is faced with the task of 
providing and assessing specific safety knowledge on a 
variety of processes and/or procedures to individual students 
throughout each laboratory hour. While this methodology is 
in itself a valid and often necessary one it does however 
lack uniformity and consistency, a serious omission in safety 
education. Secondly, and just as importantly, the methodology 
does not provide an assessment of student knowledge except 
through detailed observation, a luxury unable to be afforded 
by the instructor when surrounded with other potentially 
hazardous conditions. Finally, due to lack of feedback, little 
or no remediation can be given to the student to correct the 
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decision making process he is using to complete the task and 
so lower the potential risk involved. 

There is therefore no systematic methodology devised 
for use in the industrial arts laboratory to determine student 
safety knowledge. 

This pilot study has been carried out on the premise 
that there is need for improvement in the knowledge component 
of Industrial Arts safety curricula. Secondly the study 
seeks to define a methodology of safety education, uSing 
computer managed instruction, for Industrial Arts teachers 


that is viable in a teaching laboratory or work shop. 


STATEMENT OF THE PROBLEM 
At the university level there is no systematic method of 
measuring industrial arts student teacher knowledge of safe 
conditions and procedures that works within the usual con- 
straints of an industrial arts laboratory. Furthermore, even 
when a deficiency in knowledge does exist or is thought to 
exist, no convenient means of remediation of the deficiency 


is readily available. 


PURPOSE, OF THE STUDY 
The purpose of this study was to develop and explore the 
feasibility of a computer managed instructional system to 
deliver a series of safety tests, tests that would provide 
the instructor with an indication of student strengths, give 
the student specific remediation when required and provide 


for that student an inventory of learnings to maintain an 
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awareness of personal achievement. 
The study sought answers, based on the development and 
implementation of that system to the following questions: 

- given the need for instructor feedback will 
the computer managed instruction system, 
specifically the PLATO Learning Management 
System, provide an indication of student 
strengths, difficulties and progress? 

- given the need for student feedback, is the 
system able to provide an inventory of learn- 
ings that maintain an awareness of personal 
achievement? 

- given the need for item (question) feedback, 
is the system able, through its data gathering 
Gapabiliuy, tO provide vanvgindi cation. of overly 
Chet CalcLorvambLGQuous 2 Cems ? 

Finally, the study, given its exploratory nature, sought 
to describe the overall suitability and ease of use of the 


system for users specific to this study. 


SCOPE OF THE STUDY 
This study was delimited to the development and initial 
assessment of feasibility of a computer managed instruction 
system used to deliver several specific safety tests fora 
number of different pieces of laboratory equipment or proces- 
ses used in EDIND 211, Metals and Materials and in EDIND 412 


Metals and Materials, 1A, at the University of Alberta. 
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Furthermore, the trial and initial assessment of the system 


was delimited to a small sample of EDIND 412 students. 


ROLE OF THE INVESTIGATOR 

the roleror uhe investigator in this study was. as 
EOllows; under the direction of a senior instructor, the 
investigator was responsible for teaching course EDIND 412, 
Metals and Materials 1A at the University of Alberta Term 2, 
1982. He was also solely responsible for the initial selec- 
EPONwOr WEitLEngvangeecalting of all test 1tems, and for 
development of the strategy used to carry out the data 
analysis using the Statistical Package for the Social Sciences 
(SPSS). The variable management strategy (VMS) was developed 
by the investigator after discussions with a senior instructor 
and with the Instructional Systems Group (Computing Services, 


University of Alberta). 


PROCEDURES 
The data used in this study were collected from those 
Scucents enrol ledein EDIND 412, Term li, 1982. 
The procedures used in this study were divided into five 
Stages: 
1 the development of test material. 
2 a review of the safety tests. 
3 the authoring of the safety test material onto 
the PLATO Learning Management System. 
4 an analysis of the results. 


5 an assessment of suitability of the PLATO Learning 
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Management System and its ability to deliver 
and manage the safety tests. 

The development of the test material concentrated on the 
development of specific test objectives, determination of 
Suitable test topics and construction or selection of suit- 
able test items. The safety tests were then checked for face 
validity and correctness. of information. Authoring of the 
PLATO Learning Management System (PLM) included the develop- 
ment of appropriate PLM templates, entering of testing 
strategies and test items, item feedback and specific remedia- 
tronieor each seopic en finally ‘through ‘the: gathering: of ‘basic 
and summary PLM data an analysis of results was performed. 
This analysis was carried out to determine if overly difficult, 
overly easy or ambiguous items could be isolated. Lastly an 
assessment of Suitability of the PLM system as a vehicle for 


managing the safety tests was determined. 


SIGNIFICANCE OF THE STUDY 
Attitude, environment, inadequate supervision and in- 
adequate knowledge are, it is noted, the major contributing 
factors relating to the accident (Florio and Stafford, 1969, 
DeReamer, 1968 Heinrich, 1959, Stack, 1960, Strasser, Aaron, 
Bohn and Eales, 1973). Strasser et al. (1973), suggest that 
the "elimination of ... mishaps is based on learning and 
applying fundamental safety practices for each activity" 
(p.302). DeReamer, (1958) notes that group or mass training 


of people in the area of safety education holds an important 
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place in that training; however he feels an individual 
approach should be effected to: compensate for; individual 
differences, differing rates of learning and interests, 
arEfering: abilities and’ physical Jimitations.“(p247)-- ‘The 
Significance of this study was that it focused its attention 
on the feasibility of using a computer managed instruction, 
PLATO Learning Management System, approach to deliver a 
series of safety tests in an individualized instructional 
mode. A system that could provide, to the instructor, an 
assessment of the students' safety knowledge for the various 
metalworking equipment and processes used in the metalworking 
laboratory and correct inadequate knowledge through prescrip- 


tion for remediation when required. 


DEFINITION OF TERMS 
The following are definitions of terms used throughout 


this document. 


SAFETY EDUCATION TERMS 
Accident: 

"An incident due to error in performance by man and/or 
machine in a particular environment ... and observable in 
the form of symptoms such as pain, injury, damage and/or 
destruction" 

(Professional Preparation in Safety Education 


and School Health Education, p.2) 
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Safety: 

"A system of science-based action-oriented components 
designed to affect optimal achievement of error-free and, 
therefore, accident-free task performances which involve man- 
machine environment relationships" 

(Professional Preparation in Safety Education and 
School Health Education, p.2) 
Safety Education: 

"The process and methodology employed in conveying 
safety science subject matter ... designed to favorably 
influence decision making competencies ... demanded for 
accident-free performance in tasks involving man-machine- 
environment relationships" 

(Professional Preparation in Safety Education and 


Schoow HealthwEducatzon 7 tpie2) 


GENERAL TERMS 
Industrial Arts Teacher: 

"A teacher-manager representing a dynamic technological 
society, possessing the requisite skills current in society 
and cognizant of the rapid changes taking place in technology" 

(Ziel, H.R., Man Science Technology, p.95) 
Computer Managed Instruction: 

"A total education approach in which a computer based 
Management information system is used to support the manage- 
ment functions performed by the teacher. This totality en- 


compasses the educational goals, the curriculum, the in- 
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structional model, the teacher, and a management information 
system" 

(Baker, F.B., Computer Managed Instruction, p.14) 
Lesson: 

A PLATO type file that holds author language code for a 
CAI activity. A PLATO lesson may replace a module, a test, 
introduce a course, or be a learning resource within the 
PLATO CMI system. 

Tests 

A series of questions (items) that include text and/or 
graphic material for querying the student and for judging 
Student response. 

PLATO Aids: 

Those items available on the PLATO system and accessible 
to any user after sign-on to facilitate Instructional Systems 
Terminal (IST) usage. 

Statistical Package for the Social Sciences (SPSS): 

"A packaged program ...that will perform a specified 
set of operations, usually under the control of a simplified 
'language' or set of instructions ...specifically designed 
to compute those statistics typically used by social scien- 
tise. 

(KeckameWam, NLC pe N Hew eonadlal Hull) (©. ,obss 
Primer, p.2) 
SPSS File: 
"A distinct group of information separated from another 


by magnetic code and containing a series of contiguous mag- 
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netically stored items of information" (records). 
(Keeka, ow. ke, Nie, N.H., & Hadlia Hull, Cs, 


SPSS Primer, P.9) 


PLATO CMI TERMS 

The following terms are reproduced from: Control Data 
Corporation, PLATO Learning Management Authors Guide, Pub- 
lication Numbers 97406200 and 97406300B, St. Paul Minnesota, 
1982 
PLATO Learning Management (PLM): 

"(A) computer managed instruction system (that) manages 
testing, learning aid selection, and record keeping for 
individualized instruction as part of a PLATO system" 
Author: 

"A user who creates PLATO CMI modules and specifies 
their learning resources" 

Course: 

"A file containing a collection of modules, variable 
management strategies, feedback messages, and test controls" 
Courseware: 

"PLATO CMI courses and their component modules and 
learning resource files" 

Data Collection: 

"The recording by the PLATO system of student responses 
and other related data" 
Feedback: 


"Messages shown to the student in response to mastery 
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Or non-mastery of a question (item), instructional unit or 
module" 
File: 

"A collection of data stored within the computer system", 
(such as module files, learning resource files, course files 
and section roster files) 

Function Keys: 

"The keys provided on the right hand side of the PLATO 
keyboard that provide access to special functions" 
Gradebook: 

"A program that stores and retrieves student performance 
data" 

PS TLUCTOr: 

"As used in the PLATO CMI system: a user who constructs 
a PLATO CMI course and supervises its use" 
iStructeronal -Unst*? (Lu)? 

"The main component of a CMI module containing questions, 
a testing strategy, and a scoring procedure used to test 
mastery of a single educational objective" 

Learning Resource (LR): 

"Tnstructional materials accessible to students taking 
a PLATO CMI course and listed in the course for selection by 
the study prescription generator" 

Module: 
SAeCME eLirercontarning LU Ss; "a testing strategy, «a 


scoring procedure, feedback, and learning resources" 
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Objective: 

"The statement describing what a student is expected to 
learn and how well he will be asked to demonstrate his 
learning" 

On-Line: 

"The state of being properly signed on to the system 
through a PLATO terminal" 
Prescription: 

"A list of learning activities selected on the basis of 
the objectives not mastered in a diagnostic test" 
Question/Item: 

"The parts of an IU that include text and/or graphic 
material for querying a student and for judging student 
responses" 

Section Roster: 

"The list of students registered in a course section" 
Sign-On/Sign-In: 

"A PLATO name registered in a PLATO user group roster 
and the user-selected password associated with that name. 
Also the act of typing that name, user group, and password 
to gain entry to the system" 

Student: 

"A user that is capable of interacting with certain 
prescribed PLATO CMI modules. He can take tests, review past 
performance, receive study prescriptions and read and write 


notes" 
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Student records: 

"The performance records, statistics, and learning re- 
source assignments for each student" 
Users 

"A person signed-on to the PLATO system. His sign-on 
GCantbegthat.ofVan, tnstructor)ian authon}rton) ahstudent" 
Variable Management Strategy (VMS): 

"The controls given the instructor for governing student 
interraction with the system and its courseware. The VMS 
is designed to provide the student with a number of alterna- 
tive routes through the curriculum and can be set to present 


items sequentially or at random". 


io 


wires: ate nate | suena ba 5 
ios wea dross ott rte Rod E 282 ; 
a ean " 


| ) ari sbiveng of bong: 
ian ott outa astuge 


toshage, pimxavn, 2 
oe eat | wa mayer awe 9 


asset io sac ‘ sta 4 


dnohersg, ea 108 ed pha (rs s an 
nein eee ahi Wok «Mien 3 er’ 7 ‘eliexanaupan mind b’ 


1) 


ORGANIZATION OF THE REMAINDER OF THE STUDY 

Chapter 2 is divided into three major parts; the first 
deals with a review of current literature regarding the usage 
of computers in education, computer managed instruction and 
three related studies describing the uses of computer managed 
instruction. The second section concentrates on safety 
education and the relationship between a person's attitude and 
safety. The final section is a review of the literature 
dealing with criterion referenced testing. 

Chapter 3 details the methodology used in this pilot 
study with emphasis on the PLATO system, module design and 
development, instructional design and item analysis. 

Chapter 4 lists the findings of the study. 

Chapter 5 summarizes the conduct of the study and pre- 
sents the conclusions. The findings and conclusions are 


then discussed and suggestions for further study made. 
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CHAPTER 2 
REVIEW OF LITERATURE 
COMPUTER BASED EDUCATION 

The process by which a person learns can be divided into 
three distinct steps. First the student must be exposed to 
some material or stimuli, next he must interact with that 
material in an active manner such as writing, talking, think- 
ing or reasoning, and finally that interaction must be 
evaluated and reinforced. (Holland, L976 weed, wcrced 
Eisele, p.2, 1980). Gagne and Briggs (1974) also outline 
Similar criteria in their theory of the instructional process, 
and while Holland's view is considered by some (Eisele, 1980) 
to be superficial it does serve as a model from which tech- 
nology can be devised. 

Such technology has been devised in the form of the com- 
puter which over the ensuing years has developed from the 
teaching machine, which by todays standards should not be 
considered a computer, and in which the "dynamic component 
Of thesprogramsywas. largely illusory]. (Evans;,.19/9,*p.117) to 
responsive delivery systems which allow for input via a key- 
board and/or a touch panel. While these systems may never 
become highly sophisticated mediums for allowing complex 
learner response, they do provide a judgement of the adequacy 
of the response similar to that used in the teaching act. 
(Eisele, 1980, p.6) Whatever that judgement is however, is 
of course dependent upon the nature of the lesson design. 


Computer applications in education cover many areas, such 
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as school administration, planning, record keeping, prepara- 
tion of reports and class programs or schedules, all of which 
can be taken care of both efficiently and dependably using 


the computer. However through the use of interactive systems 


the computer is more and more being used as: 


"an instructional vehicle when some sort 
of adaption in the mode or context of instruction 
must take place between the learner and the in- 
structional program" 
(Dyery (199 25D >) 


Dyer (1972) also notes two major applications in the con- 
text of the computer as an instructional vehicle: 


- applications that include programs that 
provide drill in various subject matter skills, 
and programmed assistance for teachers in lesson 
planning or course development. 


- applications that permit completely 
individualized pupil placement, scheduling, 
testing and evaluation. 

(oe s5) 


Two major advantages of the computer in the classroom 
are again noted by Dyer: 

- programmed machinery can 'know' everything 
possible about a pupil, yet retain objectivity 
needed in order to make a decision based only on 
the criteria which are significant to the pupil's 
progress. 

- teaching programs will not 'get frustrated 
Ore tired: 

(pp.34-35) 

Caldwell (1980) writes that computer technology and its 


related instructional capabilities have an ability to: 


- provide interactive instruction where learners 
become active participants in the learning process. 


- provide alternative learning paths within lessons 
or courses 
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- offer independent pacing for individuals so 
progress can be affected at a rate suited to their 
learning style. 

- control reinforcement 

- evaluate performance quickly and accurately 
and provide data on achievement that helps im- 
prove the effectiveness of the course. 

- store information in massive amounts for 
use by the teacher and the students themselves 
in self-evaluation. 

(p.41) 

The use of the computer in the educational setting be- 
comes then a total educational environment that offers an ex- 
tensive array of learning alternatives not possible in con- 
ventional teaching environments. The capabilities of the 
computer have been applied to that same educational environ- 
ment in a number of distinct ways such as computer assisted 
instruction (CAI) where direct interaction with the computer 
may take the form of a drill-practice routine for one student, 
Or a Simulation for a group of students, or computer managed 
instruction (CMI) which uses the computer as a means of assis- 
tance in classroom management. This latter computer function 
often includes the administration of tests, the scoring and 
diagnosing of student abilities, as well as the monitoring of 
the learning sequence so as to provide the student with a 
formative evaluation and a record of progress. Whatever 
Support the computer lends however, the primary purpose is 
to assist in the individualization of instruction. (Caldwell, 


1980, p.141) 


Some misgivings regarding computer based education are 
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however being expressed. Avner (1978, p.24) notes that like 
any other medium, computer based education uses certain tech- 
niques which may be of instructional value in some situations 
but not effective in others or conversely, where potentially 
effective, they are not being used at all. Avner also stresses 
that how the medium is used is more important than the fact 
tie-medium 1s usédwat a@L1ls > (p.24) © Caldwell1*(1980),) pt 143) 

notes similar caution, in that with the use of the computer 

in education the student has been placed into a passive role 
with the teacher operating under the assumption that knowledge 
must be inserted into the heads of the learners. Programs 
must be developed, Caldwell writes, that "allow students to 
access information in a meaningfull way and one that contributes 


to meaningfull learning patterns." (p.143) 


COMPUTER=-MANAGED INSTRUCTION 

Computer managed instruction (CMI) is one computer 
development devised from a myriad of 'modern' technology and 
technical expertise. Allen (1980, p.33) defines such computer 
developments as "the use of the computer to solve instructional 
Management problems". Allen further suggests that the emphasis 
of CMI should be placed on the management aspects of the 
classroom and not on the delivery of instruction as is com- 
puter assisted instruction. He further notes that CMI has come 
to mean computer based assistance in the management of in- 
dividualized instruction, which in turn, involves the genera- 


tion of study prescriptions as well as the keeping of student 
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records. (p.34) Allen divides CMI into three basic components, 
that of a testing component, a record keeping and record 
generation component and a study prescription generation 
component. Baker (1978, p.14) defines CMI as a "management 
information system used to support the management functions 
performed by the teacher". He sees CMI as being used within 
the context of a total educational program rather than as a 
learner-oriented system, and like Allen divides CMI into a 
number of specific components: a curricular plan, an instruc- 
tional model, diagnosis and prescription management and 
reporting and the computer. (p.21) The following comparison 
can be made to those components described by Allen. (figure 


1 below) 


Figure l 


Components of CMI 
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THE TESTING COMPONENT OF CMI 

The testing component of CMI is the computer equivalent 
used to meet the needs of many human measurement operations, 
and is also adaptable to the support of the’many facets of 
testy preparativon.» \bippey’ (1974, "pt 4) “notes that "Self paced 
learning ...requires frequent use of diagnostic and evalua- 
tive instruments". Lippey further notes that assistance by 
the computer in test construction can be interrelated with a 
number of sub-systems each reinforcing the other, such as 
item statistics and directions to students toward specific 
sources for remedial study. (p.22) Allen notes (p.35) that 
CMI uses "testing for the purpose of determing which learning 
activities would be most beneficial and whether the student 
is ready for more advanced material". The major decisions 
CMI makes are based on mastery of the learning experiences 
in accordance with the principles of learning, therefore 
implicit in the testing component is the concept of behavioral 
objectives, as these provide the explicitly-written criteria 
for="mMastery Of a Subject. sin” this regard”Allen notes: 

"Since it is impractical and wasteful of 

student time to ask every question which would 

assess mastery of a defined behavior stratified 

random sampling of a ...question pool is both 


PP ereatlana oeepractical” 
(p.36) 


One consequence of behavioral objectives is that the 
learning task is broken down.into a number of elements which 
can then be rearranged into different sequences dependent 


upon the material being presented, for example, social 
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studies might. use a chronological order; industrial 
arts and science a major topic order; and mathematics a 
building block sequence. Baker (1978, p.24) identifies five 
such sequences: linear, strand, block, tree, and menu. 
Generally speaking these sequences, with the exception of 
the menu plan, are used at the elementary and secondary 
school level in various forms and degrees of complexity. 
The post-secondary level tends to favor the menu plan. Baker 
writes: 

"Under this (the menu) plan the total 

course is divided into modules or units that are 

unstructured. The student is free to select 

any module to study. When it (the module) is 

completed, he is free to select from the 'menus' 

of remaining modules" 

(p.24) 

It should be noted however, that the degree of differ- 
entation afforded to a student within any of the curricular 
plans is «quite low. Kearsly (1978, ps7) notes that from;an 
instructional design perspective, there are of course optimal 
ways to organize and present a subject, however allowing a 
student to alter this design structure would result ina 
less than optimal product. Kearsly concludes therefore that 
little or no acknowledgement of individual student differences 


exists insany ‘curricular plan without,some form,of learner 


control; 


THE RECORDKEEPING AND RECORD GENERATION COMPONENT OF CMI 
The recordkeeping and record generation component of 


CMI is perhaps the most important function of this type of 
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individualized instruction. Baker (1978, p.35) suggests 

that one of the primary causes of the emergence of CMI was 

a recognition of the unmanageability of highly unitized, rate 
of progress, individualized instructional schemes". In this 
instructional mode the students naturally spread themselves 
Over the range of the curriculum each requiring instructionally 
relevant data and each demanding considerable managerial 
attention from the teacher. 

Records must be kept on students during the instruction 
process so that subsequent decisions made by either the student 
or the teacher can be made in light of more than just the 
immediate past. (Eisele, 1980, p.3) Baker (1978, p.41) writes 
that the recordkeeping and record generation functions of CMI 
can and should be divided into cycles that reflect the teacher 
Management functions within the educational setting. These 
are: instructional, course and program. At the instructional 
level the teacher is concerned with the unit-of-instruction 
evcletathates Sa thewassagnings Ofianunit ‘of, instructioniito a 
student after determining the prerequisites, administering a 
pre-test, scoring the test and making a mastery or non- 
mastery decision, deciding on remediation in the case of non- 
mastery and the administration and scoring of a post-test. 
Once the student has completed the task the cycle will begin 
over again. The course level of management concerns itself 
with the responsibility of the teacher to have a student 
finish a unit of instruction at a reasonable rate in terms of 


the type of instructional unit being employed. This function 
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would then be concerned with scheduling, pacing and educational 
expectations, Finally at the program level the teacher is 
concerned with the educational program of a student, attain- 
ment of educational goals, curricular improvement, instruction 
of courses and the quality of educational programs. Eisele 
(1980, p.3) notes that the most important of these functions 
is the desirability of providing an inventory of learnings 

and experience to individuals to maintain an awareness of 
their own status. Allen (1980, p.34) writes that CMI gen- 
erates large volumes of data which must be manipulated and 
stored in such a way as to serve two primary functions, that 
of indicating student strength, difficulties and progress and 
validating test items, objectives, and instructional materials. 
Hazen (1980, p.281) noted that the recordkeeping function of 
the CMI system is also useful as a research tool in terms of 
CMI evaluation. Data collection is, she writes, "relatively 
easy, inobtrusSive and has the added advantage of being 
collected over a long period of time". The major problem 
however with the recordkeeping and record generation com- 
ponent of CMI is as Baker notes (1978, p.303) its limitation 
to symptomatic diagnosis. The information given is at a 

basic level of management and consists of objectives passed 

or failed, test scores and dates on which tests were given or 
assigned. None of this information is integrated in a 
diagnosis and therefore the teacher is left to interpret the 


data on a judgemental basis for his students. 
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THE STUDY GENERATION COMPONENT OF CMI 

Kalisch (1978, p.37) writes that the purpose of the CMI 
Study generation or prescription component is to direct the 
Student to areas that only require his or her attention or 
to areas of the students' interest. 

"prescription frequently direct individuals 

to specific pages or chapters of books, to 

video, or audio tapes or to other instructional 

materials" 

Allen (1980, p.37) notes that the computer reading the 
data generated by the tests given, as well as any additional 
data provided by the instructor, can compare diagnosed needs 
with a list of instructional materials, then select and pre- 
sent a study prescription to the individual student. It is 
noted by both Allen (1980) and Baker (1978) that any system 
which does not generate study prescriptions is not CMI and 
that both diagnostic and prescription procedures should be an 
integral part of any CMI system. "The level of diagnostic 
information given by any CMI system is considered to be ata 
Father 'erude' level-of detail" (Baker, p.31) The computer 
in no way, it is noted, Baker et al., makes a diagnosis of 
why the student has not mastered a given objective, it only 
indicates, thesstatus of the student relative “tothe scurricu- 
lum. 

Two types of prescription are generally used in most 
CMI systems, the forward and the remedial. The forward, 
based on the student's mastery of a given unit, will allow 


that student to select the next module on the menu. The 
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remedial prescription is based on the student's non-mastery 
of a given unit and takes two forms, that of having the 
student re-study the material or assigning the student to 
some educational activity or resource. Baker (1978, p.32) 
has noted that most "CMI prescriptions tend to be remedial 
in nature" and most prescriptions, except those especially 
contrived to fit a single objective, cover several objectives. 
One extension of the concept of study prescriptions is noted 
by Allen (1980, p.38) and is used on the PLATO system. Here 
the percentage of times each available resource has been 
found to bring the student to a mastery status is recorded. 
The system then references this 'success' information in 
order to select the resources having the best potential for 


meeting a student's needs. 


THREE RELATED STUDIES 
The following three studies describe the different ways 
in which CAI and CMI have been used in education. The three 
Studies are: 
1 The Wisconsin System for Instructional 
Management (WIS-SIM) 
2 A Computer Assisted Study Management System 
(CAISMS ) 
oan Alternative Curriculum For Computer 


Literacy Development (ACCOLADE) 
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THE WISCONSIN SYSTEM FOR INSTRUCTIONAL MANAGEMENT 

The Wisconsin Ssytem for Instructional Management 
(WIS-SIM) Spuck and Bozeman, 1978, became operational in 
1974 using the Madison Academic Computing Center's UNIVAC 
1108 computer. The system was pilot tested from 1975 through 
1977 when the project was concluded, and served a total 
of 1350 students. 

The WIS-SIM program was a computer-based management 
information system designed to aid teachers with record- 
keeping and the other clerical tasks involved with the in- 
dividualization of instruction. The program, Spuck and 
Bozeman, note,-(p.33)awas. noty~curricular but, had, as,ats. basic 
premise the collection and processing of student information 
that was "directly applicable to instructional decision 
making". The underlying assumption of the program was that 
when appropriate information was supplied to teachers, students 
and administration, the quality of decision making improves 
and can be evaluated. The WIS-SIM program supported a com- 
prehensive system of education designed to produce higher 
educational achievement by providing an individual rate of 
learning and learning style. 

The system was also an interactive one in that the user 
could communicate directly with the computer via terminals 
located in various schools. Operators could enter information 
such as student achievement data, new student enrollment, 
deletion of students from the school roster as well as 


comments regarding the students and their test scores. Reports 
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from the system took the form of student performance profiles 
either individual or group, student listings, instructional 
grouping recommendations and a program of study recommenda- 
tions. 

In their evaluation of the system, Spuck and Bozeman, 
noted that the system rated quite highly in terms of useful- 
ness- Also noted was teacher planning time. Actual time taken 
to complete a task did not change, however the process itself 
became more efficient and effective and more was accomplished. 
The use of the WIS-SIM program also indicated, during the 
evaluation period, a positive trend in student achievement. 
Spuck and Bozeman suggest however that this finding should 
be interpreted with some caution as a large number of other 
factors in the school environment may have contributed to the 


trend. 


A COMPUTER BASED STUDY MANAGEMENT SYSTEM 

The Computer Based Study Management System (CAISMS), 
ANGGESON,. Teh. a Anderson, R.C., Dalgaard, B.R., Wietecha, G.J., 
Emidvce AWeb Paden DoW. ,.SmOCck sina kRa, Allessi;7S iM. 
Surber, J.R., and Klemt, L.L., 1974, was designed to maintain 
attentive study of instructional materials in an introductory 
level, college economics course and served 70 students. 
CAISMS was delivered using a PLATO IV computer terminal. 
PLATO, Anderson et al. (p.38) note was chosen for the project 
for two reasons: first its capacity to present material 


in a pictorial, diagramatic and graphical format and second 
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in a response mode, "PLATO ...permit(s) fairly sophisticated 
treatment of freely composed natural language answers". 

The system itself was designed around existing materials 
which primarily consisted of the textbook being used in the 
economics course. A number of summarizing statements on the 
most important points presented in the text were prepared 
which then served as the basis for the generation of the test 
items. The test items were then formulated using either para- 
phrases of the language of the text or examples of the concepts 
different from any appearing in the text. By using this 
method it was felt that the student would have to have a 
more than superficial understanding of the text material to 
answer the questions correctly. 

The system itself required that the student use it for 
only approximately a fifteen minute period per hour of study- 
ing, typically during this period the student would sign on 
to the PLATO terminal and be provided with a specific study 
assignment, the student then leaves the terminal to study the 
assignment given. As he signs-off, a seven minute delay is 
placed on the system, during which time he cannot sign back 
on. Once the student has completed the assignment he then 
goes back to the terminal signs-on and is given a test on 
the assigned material. No feedback is presented until all 
the items in the item pool have been presented and answered. 

The criterion level for this program was set at 75 per- 
cent and if a student did not achieve this he was required to 


re-take the test. Prior to the re-take however a ‘lockout' 
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waS again placed on the system during which time the student 
was to re-study the material he missed. If the student 
achieved mastery on the first test he was then routed through 
a pass routine to a new assignment. 

Anderson et al. found in their evaluation of CAISMS that 
Fe eEunetioned well “in*the structure of “an ‘ongoing “college 
program. They also found that student study habits improved 
and that students using the system scored higher than those 
students not using the system. The authors do note however, 
that the evaluation information should be considered "only 
Suggestive" and that several modifications to the system 


would have to take place to increase its impact. 


AN ALTERNATIVE CURRICULUM FOR COMPUTER LITERACY DEVELOPMENT 

An Alternative Curriculum for Computer Literacy Develop- 
ment (ACCOLADE), Denenberg, 1980, was implemented on a pilot 
study basis with ten learners and two teachers. The system 
was designed to offer a curriculum system that allowed 
students to educate themselves about computers. (p.56) To 
acquire the computer literacy knowledge the learner first 
had to use the computer to search through a number of 
literacy topics, then through the use of a number of responsive 
display devices the student was shown the structure of the 
LOCC: 

"to gain the skills of logical thinking, 

problem solving and learning confidence... 

During the learning session the learner works 

with the other learners ...using a programming 


language to solve case study problems associated 


with the learner's discipline and interests" 
(pe o7) 
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If a learner wished to accredit the experience to his 
record he could self-administer a range of criterion refer- 
enced tests using a CMI PLATO system. The CMI system was 
set up to handle testing, grading recording, reporting, 
diagnosis and prescription, as well as monitoring test 
progress, learner-computer transactions, issuing management 
reports and performing continuous item analysis for each 
question so that poor questions could be improved or eliminated. 
The system was considered to be reasonably successful in that 
it demonstrated an alternative and quite effective teaching- 
learning device. (1977, p.72) Denenberg also noted that the 
PLATO terminal provided an excellent communication medium 
between the learner and the computer, affording the learner 
an easy-to-use, highly controlled search tool. 

The CMI reports obtained from the PLATO system were quite 
comprehensive. The total number of visits to each of the 
components of the system was recorded as was the specific 
learner path through the program. The PLATO system also 
provided data upon which decision making could be based. 

Such decision making centered around the quality of the 
questions being answered: was the question too hard, tricky, 
ambiguous, highly positive or negative discriminating? 

The system as a whole was noted (p.172) to have demon- 
strated a reasonably effective teaching machine that allowed 
the learner the educational alternative of taking responsi- 


bility for effecting his own education. The system was also 


eb ao eben 


a aes amine di * 


RL a ee ere a 
fee a ED Pera | ey | 


Lee 
. ae tae eee ee wits pn mi ; i 
hey by ‘ ree e. ai ie ce Lene Oy: Paso6} “— 


=wae Sonne spelen La + hes oie, aa equaiqwre sa cs fw tee 


meee) a Seni ean eta ade inn, or a, vent one: pk : 
mS ee ea im ia eile en 2 
res Riga we Diane 4cc08 cic a i a ity Soe oor , hens 3 oe Bad 
 eerseatal en gt iopne oe " 
fap 


tee r “hw rab ai wa eee ehaiabrbinsy ‘gg nota \aikio 
or tig | one si nh meer’ * aie “a bias wai do. item 3 wad: feat 


wig s fio? ie oy “eh. te . 


coe i a 


| Oe, Lone age wee 4e Laub 


outa. ails Su: + onal 3 ait .3 
i Seah hoe db Hoon. Lacan 
a a ‘ei eo weft “ por 

‘oat ae oe bite, oe anitecha 


re 


a tea eae ait, ow hte i 
ood aiid by ke stron 
) 


Uo Rune Oh el 


36 


noted to be moderately reSponsive to learner and teacher 


needs and their requirements. 


SUMMARY 

The use of computers in education points to computer 
usage serving two important functions in the total educational 
process: that of instruction and management. These functions 
allow the student to effect their own learning and allow 
teachers to collect data that provides viable decision making 
capabilities. 

The Wisconsin program describes a CMI system using a 
UNIVAC 1108 computer that provided a management information 
system to support decision making on behalf of students, 
teachers and administrative staff. The system was found to 
be reasonably successful. The CAISMS program was designed 
around a totally different concept and interactive computer 
device. In this case a PLATO IV terminal was chosen to 
provide, through its CMI learning management system, a series 
of tests based on the students' text material as an aid to 
learning and improving student study habits. 

The remaining study ACCOLADE, like the CAISMS system 
also used PLATO. However in this case a PLATO Learning 
Management (PLM) System was developed to allow the student 
to effect his own learning through a series of criterion 


referenced test. 
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SAFETY EDUCATION 
"In the development, of 'safety awareness' 

the aim is to condition a person's mind so that 

whenever there is one element of danger present, 

a sixth sense will cause him to react in the safe 

way" 

(DeReamer, 1958, p.16) 

The conditioning of a person's mind to develop a 'safety 
awareness' is generally categorized into three areas of 
concern: accident prevention, safety programs and the funda- 
Mencals, OL Sate living. Florio and Stafford (1956,.p.11) 
note that in the area of accident causation the primary 
factors are, 1) unsafe behavior, 2) inadequate knowledge, 

3) insufficient skill and 4) improper attitudes and habits. 
Florio and Stafford further note that several of these factors 
usually operate together and it is not always possible to 
isolate one particular attitude or characteristic responsible 
for the accident. Strasser, Aaron, Bohn and Eales (1964, 
p.302) feel the human element is the most Significant factor 
related to accident causation and they like Florio and 
Pearrord List four factors in that: area. These. are: 

1) attitude, 2) environment, 3) inadequate knowledge and 

4) inadequate supervision. They feel that the "elimination 
of such mishaps is based on learning and the application of 
fundamental safety practices for each activity". Safety 
programs they note (p.107) must be approached in a positive, 
business-like manner and a well balanced program should be 


developed to take advantage of all well known and proven 


principles and techniques of accident prevention. Among the 
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basic elements they stress are: the keeping of accident 
records and reports, enforcement of safety rules and regula- 
tions, education, training incentives, evaluation and research. 
Heinrich (1956, p.113) feels safety programs must be based 

on all the “pertinent facts of accident causation" to which 

he suggests there are four types of action available: 1) 
engineering revision, such as the removing or guarding of a 
potential hazard, 2) persuasion and appeal, which includes 
instruction, 3) personal adjustment, and 4) discipline. 
Heinrich also notes that the individual person as a result 

of proper supervision, safety education and personal ini- 
tiative can correct his own safety practices and can also 
maintain safely a good part of his physical environment. (p.45) 

DeReamer (1956, p.47) notes that group or mass training 
of people in the area of safety education does in many cases 
hold an important place in that training, however he also 
feels an individual approach to instruction must be effected 
to facilitate individual differences, a person's rate,of 
learning, his interests, abilities and physical limitations. 
"Personalized safety training can be specific, group training 
must be general". 

Worick (1975, p.25) also notes criteria similar to those 
of Florio and Stafford. These are: 1) inadequate knowledge, 
2) insufficient skill, 3) environmental hazards, 4) improper 
habits and attitudes, and 5) unsafe behavior. Proper know- 
ledge, notes Worick, enables the individual to recognize and 


evaluate dangerous situations. This combined with their 
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personal skill is an important determinant in accident pre- 
vention. Like Heinrich, Worick Suggests the environmental 
hazards should be engineered out of society and a person be 
educated=to rid himvof faulty habits: and attitudes®which’ are 
prime accident producers. Unsafe behavior, he notes, is the 
end result of a person's failure to develop proper habits, 
attitudes and knowledge regarding safety. "Safe behavior 
entails responding correctly under all circumstances and 
avordimguen shagheriskesituati ons") (pr25)eworicke further 
notes (p.45) that "teaching methods must be positive in nature 
with the constant reinforcement of safety knowledges being 
standard procedure". 

For Worick, safe behavior is the optimum performance 
of a progression of seven processes which are as follows: 
1) identification, the hazard is first identified, 2) assess- 
ment, the hazard is assessed in terms of risk, 3) decision, 
once the risk has been assessed the decision for safe per- 
formance is taken, 4) performance, safe performance takes 
place, 5) evaluation, after the performance has taken place, 
the action is evaluated, 6) modification, once the performance 
has taken place and been evaluated in terms of safe behavior 
the action might need modification to make it even safer, 
ands 7) acolication, at this point. in the process experience 
is applied to the task so as to reach the desired goal of 
safe behavior. 

"We know that behavior may be changed as a 
result of an increase in man's knowledge, 1m- 


provement in his skills and modification of 
his attitudes" (Thyerson, 2976) p.97) 
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Thyerson further notes (p.97) that safety rules as such 
should not be emphasized as these lie at the lowest level of 
cognitive learning: a student may know the rules but never 
apply them. To influence safe behavior, concepts must be 
developed that lead to good 'safe' decisions being made. 
Safety concepts must "equal the major mediating variable in 
human decision making and purposive behavior" (Woodruff, 
cited Thygerson, 1976, p.113). It is therefore necessary to 
replace the safety rule with the conceptual statement. 

Strasser et al. (1964) feel that only through safety 
education can a safety philosophy be developed in the student. 
Behavior they note: 

"is controlled by values, attitudes, and 

habits which in turn, are developed and modi- 

fied by the acquisition of knowledge and 

experience" 

(pe 5.) 

Florio and Stafford (1956, p.20) suggest a similar 
approach with emphasis being on the understanding of the many 
hazards one must encounter in daily life, the development of 
attitudes that predispose a person to adjust properly to his 


environment and mastering skills that enable a person to 


cope with potentially dangerous situations. 


ATTITUDES AND SAFETY 
The concept of an attitude is closely allied with that 
of emotion and is most often manifest when a person responds 


in a predictable manner. 
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"Attitude is generally conceptualized as 

a learned predisposition to respond in a consis- 

tent evaluative manner toward an object or a 

Class Of objects! 

(Miider) «1969 ,*pe3) 

Many attitudes are latent; that is, the attitudinal 
response is dormant until required by that person, who, when 
given the appropriate stimulus, brings the attitude to the 
fore. The behavior exibited at this time is then in accord- 
ance with the feelings of that individual. Attitude is con- 
Sidered by many to be the most important concept of safe 
human behavior. Stack and Elkow (1966, p.39) note that many 
safety educators "consider them (attitudes) more important 
than either knowledge or skills". Brody (1962, cited Stack 
and “Elkow, 1966 7,-p.39)" notes’: 

"I believe we can state that the relatively 
specific attitudes that need to be developed 

are those that convince the individual that 

accidents are hardly accidental ...that they 

are for the most part personally caused, and 

they they are controllable" 

Bilorilo, and Stattord (1956, .:p.3) feel attitudes in-= 
fluence one's fundamental concept of honesty, work and personal 
responsibility. They feel that attitudes "determine the 
degree to which success is achieved in making adjustments to 
environmental conditions". Strasser et al. (1964) note 
that when safety is defined as a function of human behavior, 
the importance of controlling this behavior through the 
development and modification of attitude becomes very im- 


portant. The role attitudes play however, becomes very com- 


plex and each situation brings many conflicting and support- 
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ing attitudes together which interact and determine behavioral 
response. Brody, et al. (1962) note that a number of sug- 
gestions have been put forward for the modification of atti- 
tudes toward safety and the determination of behavioral 
response. These suggestions include logical appeals to 

place safety education on the basis of reason, and the 
admonition and exhortation of the persons involved in the 
safety program to behave ina safe manner. However Strasser 
et al. (1964, p.188) suggest that little or no success has 
been achieved by any of these methods as they appear to 
inhibit rather than create any true behavioral change. They 
further note that "the principal aim of safety education 
..-.centers around the modification and development ...of 
acceptable attitudes" (p.189) and like Stack and Elkow (1966, 
p.189) feel a problem solving technique should be developed 
toebringeaboutethatimodaficationsandsadevelopment,of.acceptable 
attitudes. Also noted is that unlike other areas of educa- 
tion, safety education should be within the interest and 
capacity of the teacher as teacher enthusiasm is an essential 


Patt Of:stheainstructronal, process: 


SUMMARY 
[ft ie now widely recognized that human traits are ex 
pressed in such terms as perseverance, curiosity and level 
of aspiration, as well as being a function of heredity and 
environment. Most writers in the field of safety education 


note that “evidently accidents are multi-causal due to 
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factors which are present at sometime in some people in 
certain settings. The concensus of opinion towards acci- 
dent prevention points to a need to instill in people the 
concept that the development of protective skills rests 

with them, that human traits tend to increase human vulner- 
ability to accidents which are centered around everyday 
environmental hazards, inadequate knowledge and insufficient 
skill to work at an optimum level in that environment, so 
creating improper attitudes and working habits. The authors 
feel that the elimination of the accident rests with the 
learning and application of fundamental safety practices for 
all human activity. 

Attitude, it is noted, (Miller, et al.) is manifested 
by overt benavior and is often latent until recalled in 
response to the feelings of that person. Attitude modifica- 
tion is considered to be the most fundamental concept in 
safe behavior development. (Stack and Elkow, et al.) A 
number of methods have been suggested for modification pur- 
poses such as fear, humor and exortation. Most however, have 
been unsuccessful. What is suggested are the more standard 
teaching practices that allow a student to practice the safe 
ways demonstrated by the teacher, or a problem solving tech- 
nique where the student moves through a set procedure identi- 
fying and assessing risk and making decisions on actions that 
must be taken to perform in a safe manner. (Strasser, et al., 


1964, Worick, 1975) 
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CRITERION REFERENCED TESTING 

Criterion referenced testing is used to "ascertain an 
individual's status" (Popham and Husek, 1979, p.133) with 
respect "to a well defined body of knowledge and behaviors" 
(lorgan, 975, D.1) ~~ eihis standing: 

"is often represented in terms of a measure 

of performance (competency or functioning) based 

on the individual's test data, which is often, 

but not always independent of the performance 

of others that might be tested with that in- 

dividual" 

(Ibid) 

The individuality of the criterion referenced test then 
presents a number of problems in its analysis. Like the norm 
referenced test, items that do not discriminate between the 
knowledgeable and less knowledgeable learner must still be 
identified. However for the criterion referenced test the 
discrimination indices must be modified. Popham and Husek 
972, op .l 59) note: 

"an item which doesn't discriminate need 

not be eliminated. If it reflects an important 

attribute of the criterion such an item should 

remain in the test" 

However they do caution that: 

"discriminating indices are little more than 
warning flags and one still must use common sense 

in weighing the worth of an item as a negative 

discriminator’ 

(p.140) 

Because of the criterion referenced tests concern with 
the relationship between a person and his own level of per- 


formance and not the performance level of others, traditional 


measurement practices to determine reliability and validity 
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as well as interpret test results are not applicable. 


RELIABILITY 

Reliability, defined by Stanley (1971, cited Thorndike, 
1971, p.356) as the “tendency toward consistency from one set 
of measurements to another" is as necessary in the criterion 
referenced test as in the norm referenced test, however 
determining reliability in the criterion referenced test 
becomes a difficult process. Ferguson, (1976, p.413) notes 
that the purpose of any (norm referenced) test is differentia- 
tion of individuals, he further notes that: 

"Clearly a test on which every individual 

made a ...perfect score ...would serve no useful 

purpose" 

The construction therefore of a norm referenced test 
attempts to ensure that the test scores have a fairly large 
variance in relation to the number of items the test contains. 
However the criterion referenced test is not intended to dif- 
ferentiate between individuals, and a perfect score would 
serve a useful purpose. Using traditional measurement tech- 
niques however the test would have a zero internal consistency 
estimates.) Popham and: Husec (1972, 6.137) note that 2t is 
not clear what should replace the traditional indicies of 
internal consistency, and suggest estimates comparable to the 
standard internal consistency formulas, such as that suggested 
byeHuset sand Sirotnik (1968) -or ingenious indices to vreflect 
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Hively (1974, p.8) suggests the notion of reliability: 
"has to do with the accuracy with which 

one can eStimate the probabilities of correct 

performance within a concrete domain" 

He further suggests that in order to be able to estimate 
the probabilities of correct performance the goal of domain 
referenced testing is to "create an extensive pool of items 
that represent in miniature the basic characteristics of some 
important part of the original universe of knowledge". (p.8) In 
this way a student who is able to respond correctly to these 
items will be able to generalize to the field. (p.6) Baker, 
(1974, p.11) concurs and notes that: 

"A domain consists of a subset of knowledge, 
skills, understandings or attitudes where the 

essential attributes of the context which the 

student is expected to acquire and the behavior 

through which he or she is expected to demon- 

strate such acquisition are carefully described" 

Morgan (1979, p.5) notes however that "it may not be 


possible to determine with absolute certainty that the level 


of performance on the domain of tasks. 


VAGDLDI DY 

"The most meaningful measures of the value of a test are 
Ptsomvalvdities” .(Lord, and Novick, 1968). Validity 1s defined 
as "the capacity of a test to predict some specified behavioral 
measure (or set of measures) other than itself" (Cattell, 
1964 p.338) or the "accuracy of a specified prediction or 
inference made from a test score" (Cronbach, 1971, p.443). 
Like reliability, the procedures for the assessment of 


validity are based on the variance between the test score and 
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an observed criterion and are best applied to the norm 
reterenced test. iIn the case of the criterion referenced 
test, Popham and Husek (1972, p.138) suggest validation be 
conducted in terms of the adequacy with which the criterion 
referenced measures represent the criterion and therefore a 
content validity approach is more suited to such tests. Con- 
tent validity, Cronbach (1971, p.444) notes is evaluated by 
showing how well the content of the test samples the class 
of situations or subject matter about which conclusions are 
to be drawn. Hively notes (1974, p.8) that: 
"Validity has to do with the success of 

generalization from performance on a concrete 

domain to performance in the larger universe 

of knowledge from which the domain was 

generated" 

The determination of validity in the criterion referenced 
test becomes then a determination of how well the test pro- 
duces a clarified description of what an examinee's perform- 
ence=silgqniri1es: (Popham, 19767" prlo>) =To*doethis;, -cest~items 
should be homogeneous, that is the items should all measure 
the same variable or objective. (Davis and Diamond, 1974, 
erred Morgan, 19745 p¥5)*PCronbach (19717 ps457)=notes*how- 
ever that: 

"Judgement is required in establishing the 

accuracy of items as well as their relevance 

to the universe specifications.. Test items 

should stana up under Criticism from experts 


inthe Subject matter tested” 


He@ further notes’ that: 


"an item ...should be consistent with the 
latest literature on the subject known ... not 
just with some out-of-date superficial conclusion 
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presented in a textbook" 
(Ibid) 


Secondly tne valudity of the criterion referenced test 
can be determined by its functional validity or the accuracy 
with which the test satisfied the purpose to which it is being 
Pulte Opham, 19/78, pato9) A set of Ttems or tests, must, 
Suggests Popham, make accurate predictions. Finally, validity 
of the criterion referenced test can be determined by: 

(the) "accuracy with which the evaluator 

selected a particular domain to serve as the 

indicator of learner status with respect to a 

more general dimension" 

(Popham, 1978, p.164) 
Domain selection is, notes Popham (1978, p.164) similar 


to those procedures used in the construct-validation approaches 


employed by the norm-referenced test. 


SUMMARY 

Criterion referenced testing is used to determine a 
person's status and is represented as a measure of performance 
independent of the performance of others. Because of this 
independence traditional measurement indicies cannot be used. 
What should be used to replace the traditional techniques 
is not clear, ideas range from estimates comparable to the 
standard internal consistency formulas for use when determin- 
ingureliability to estimating the probabilities of Correct 
performance within a concrete domain. 

The determination of validity in the criterion referenced 
test also presents a number of problems. Here it is suggested 


that validation be carried out to determine the adequacy of 
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the criterion referenced measures representing the criterion, 
or an evaluation be conducted to see how well the content of 
the test samples the class of situations about which con- 


GLusions are’ tobe’ drawn: 


CHAPTER 3 
METHODOLOGY 
PLATO CMI INSTRUCTIONAL DESIGN 

PLATO is an interactive computer-based educational (CBE) 
System developed to enable instructors and courseware designers 
to create and deliver instruction. 

When this study was conducted, PLATO at the University 
Alberta was uSing a Cyber 170/720 computer having 98,000 
words of 60 bits each of central memory and 1,000,000 words 
of extended semi-conductor memory. (PLATO Aids, The PLATO 
Information Systems Terminal, Control Data Corporation, n.d.) 

The PLATO Information Systems Terminal (IST) provides 
interaction between users and the computer, it is also cap- 
able of displays, animation and selective erase. The terminal 
is the primary device by which students, instructors and 
authors interact with the computer. 

The primary means of input to the terminal is via the 
keyboard. However users can also enter information using a 
touch panel. The touch panel contains a 16 by 16 grid of 
electrodes defining 256 touch-sensitive areas. PLATO 
Demonstration and Evaluation, Computing Services, The 


University of Alberta, 1978) 


PLATO LEARNING MANAGEMENT 
The structure of the PLATO Learning Management System 
(PLM) is centered around the Group, a Curriculum, Modules, 


Instructional Units or Objectives, Question Pools and a set 
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of Learning Resources or Remediation Prescriptions. (See 


Figure 2, PLM Structure, below). 


Figure 2 


PLM Structure 


MODULE MODULE MODULE 
INSTRUCTIONAL UNIT INSTRUCTIONAL UNIT INSTRUCTIONAL UNIT 
QUESTION POOL QUESTION POOL QUESTION POOL 
LEARNING RESOURCE LEARNING RESOURCE LEARNING RESOURCE 
MODULE MODULE 
INSTRUCTIONAL UNIT INSTRUCTIONAL UNIT 
QUESTION POOL QUESTION POOL 
LEARNING RESOURCE LEARNING RESOURCE 


The PLM individualized instructional model is designed 
around a single module of which there are many within each 
course. After the student signs on to the system and is 
presented with a menu (of modules) or router lesson, he then 
selects the module he wishes to 'test out on'. Once that 
selection is made, the student is then presented with a 
Series Or Questions .(items) in the form of a test. If the 
test or module is not mastered, the student is given a 
selection of learning resources (LR's) particular to that 
module. (The LR's given the students in this study were in 
the form of page references taken from the textbooks used 


in the Metals and Materials courses. These assignments were 
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not authored onto the PLATO system.) The studying of the LR's 
help the student continue toward mastery of that module and 
subsequently the course. After the student has studied the 
LR's and feels ready to demonstrate his learning, he can try 
the test again. (PLATO Learning Management Authors Guide, 


Controlt Data Corporation, 1982) 


PLM DATA 

The PLM system collects data into a student data file 
attached to a PLATO group. There are three types of data 
collection; basic, extended and summary. Basic and extended 
data are used to collect data from the test items presented 
to the student and only one or the other may be used at any 
one time. Test summary data gives overall test performance 
and can be used independently of, or with basic or extended 
datar,collection. “Data collection, can be turned off, setito 
collect data from each of the students first test, or set to 
collect data from every test. (PLATO Aids, The PLATO In 


Information Systems Terminal, Control Data Corporation, n.d.) 


VARIABLE MANAGEMENT STRATEGY (VMS) 

Variable Management Strategy (VMS) is designed to provide 
for the student a number of alternative routes through the 
curriculum. The VMS can be set to present sequentially or 
at-ranaom, both the “Instructional Units (IU"s) and/or the 
items in the item pool. (PLATO Learning Management, Authors 


Guide, Control Data Corporation, 1982) 
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PLATO PILOT STUDY INSTRUCTIONAL DESIGN 

The CMI curriculum developed for this pilot study com- 
prised twelve (12) safety modules each designed to complement 
the work being carried out by the instructor in the labora- 
tory and each intended to expand student knowledge in the area 
of metalworking aeeuen 

The modules themselves were delivered to the students 
via the PLATO Informational Systems Terminal. Each module 
was set up in the PLATO Learning Management Systems (PLM) 
by the Instructional Systems Group, Computing Services, at 
the University of Alberta. The Variable Management Strategy 
(VMS) within PLM selected for use in this study was to have 
the PLM system, after the student had selected a module, 
randomize the instructional units (objectives) within that 
module. The PLM system then selected one item from the pool 
within each instructional unit and presented that item to the 
student. The criterion level was set at 100 percent. The 
number of instructional units per module ranged from a low 
of ten (10) to a high of sixteen (16) with the average number 
of units being thirteen (13). The number of items in each 
item pool ranged from a low of two (2) to a high of seven (7). 
The pilot study used both basic and summary data collection 
which was set to collect data from every test the student 
took up to and including the first time he achieved mastery. 

The student path through a PLM module is shown in 
Figure 3, page 55. The student would sign on to the PLATO 


system and would be presented with the router lesson or menu. 
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From this menu he would select the module he wished to take, 
and was reminded, as an introductory message, that the cri- 
terion level was set at 100 percent. If the 100 percent level 
was not achieved, remediation in the form of page references 
to the Metals and Materials course textbooks were presented 
at the end of each testing sequence. After the introduction 
the PLM system randomized the various Instructional Units, 
then selected one of the items from within each item pool 
for presentation to the student. When during the test the 
student answered the item correctly he was presented with 
the feedback message 'correct', if the item was answered 
incorrectly he was presented with a feedback message stating 
he EGR the answer was considered incorrect. This pro- 
cedure continued to the end of the test. 

At the conclusion of the test the student was presented 
with either a mastery feedback message congratulating him on 
a perfect score and suggesting he go on to the next module, 
or a non-mastery message telling him he did not achieve what 
was expected and would now be expected to read the text re- 
mediation presented to him. At this point the student was 
locked out of the PLM module for a period of fifteen (15) 
Minutes. The student now had a choice, he could continue on 
the system and work on another module or he could sign off the 
system, complete his assignment and retake the test at a later 
time. On test retake the student was compelled to take all 


of the test again, he could not answer just those objectives 


Pa RAs = 0 
Yaw he a Hy; 


/ i i i 
iat ae ; bk? 
Leen drow oid dnd « ae 


-_ wana ae . > Me 98 wads 
aM o ; ; pe a ; 


bite qq octiaew eit node nea v2: me. i | edit ban eda 
uaa wea ine oan ion ‘ga0e ony. to tne 
i - f , AG cen ei yy ow *s sta 5 pat vai ot . “the ‘ sohiant Ca ll 
ie, cakes oe ni sich a cit ee meen bexselae 6 
eat $2 a canes oriite : sida | ay er ati’ ¥ “a “ ide ot $ doidagnensag.| 
step den Be ee iy , F ane ‘ 4s i 2 43 teens wt Habit 
ee ie Me ee ie 4) 037a3705 = 20m x searinahtes 


_~ 


ee ae detain om _ sows we ae 
| | * \liaamal phe te hee end cigs ‘Seunbadon tect 
‘fechas 1 Bey eerie nsw irk 8 ae ty rotmiemns eee 46 
abe vie a. pant re tus a erence " seesaert ‘Romie exe Kinin zett hel natty bai 4 
eiakaaboue gana ate nh tile gt iidenowees e ” sxo98 ia 
she: aw anit sion Wk. se ‘kn el ) exanenmns —— 
a] Ano aul iain Dy 9 ik wine in cow ‘na laiatpets il 
a ~ 2 teats ‘ite ation bil’ <e ot asm, soizatins 


i, 


aity PRY Die te | ea ih albus ena me 


my 


ee ae naar Syn 0 


an a 


Do 


Figure 3 
Student path through a PLM module 
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he failed to master. However because of the randomization 

of the IU's and the subsequent selection of one test item 

from each LU pool the student stood little chance of receiving 

the same test. Also available to the student after sign-on 

and during the test period (but not. during the testing it- 

self) was a router to the gradebook. The gradebook 

allowed the student to see his progress to date which included 

the number of times he had taken a particular module before 

mastery, each module he had taken, his test scores, the score 

for the test he was presently working on, if he signed off 

in the middle of a test and the module he should work on next. 
Figure 4, PLM strategy for the student taking a test, 

following page describes the PLM strategy employed as the 

Student was taking a test. The PLM system first presented the 

menu from which the student would select one module. The 

instructional units in each module were then randomized and 

one test item from each of the item pools within each IU 

was then selected and presented to the student. If the student 

failed to answer the test item correctly he was given feed- 

Dacketouthatwertect and,a ‘flag’ wastset. If the student 

'mastered' the item he was presented with the appropriate 

feedback and the system would set an ‘increment counter P'. 

At the end of the testing sequence the system would then 

check for the number of flags indicating non-mastery. If no 

flags were set, the student was routed through a pass routine, 

if flags had been set they were 'picked up', a non-mastery 


message was presented, followed by a prescription for remedia- 
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Figure 4 
PLM strategy for the student taking a test 
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ELON. 
MODULE DESIGN AND DEVELOPMENT 


The modules used in this pilot study were developed from 
two sources, one a series of safety tests designed and piloted 
by the Department of Industrial and Vocational Education at 
the University of Alberta, and two, a series of EDIND 412 
class assignments in the writing of appropriate safety test 
items. 

For the purpose of this study a list of twenty-one (21) 
metal-working subject areas such as the Drill Press and the 
Engine Lathe were drawn up then placed into the approximate 
order that the student in the University laboratory would be 
using them. Those tests designed and piloted at the University 
and those written by past EDIND 412 students were then fitted 
into the subject area list and developed first, then as time 
permitted, subsequent modules were developed up to the time 
of the actual pilot study. After fitting the already ‘developed 
Material into their various categories the next task was to 
devise a terminal objective and a number of instructional 
units for each module. Shown in figure 5, Terminal Objective 
and Instructional Units for the Drill Press Module, following 
page, is the terminal objective and list of instructional 
units developed for the Drill Press module. 

Once the task of devising both the terminal objective 


and the instructional units was complete, a PLATO author 
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Figure 5 
Terminal Objective and Instructional Units for the Drill Press 
Module 


Upon completion of this module the student 
will be able to list the safety precautions 
and (demonstrate on the Drill Press) the 
safe working practice associated with: 

- speed selection for drills 

- chucks and chuck keys 

- eye protection 

- drilling small work 

= SCOn:iningadong Nar 

- personal clothing 

- drilling long pieces of material 

- removing chips 

- removing sleeves, sockets and drill chucks 
- holding work 

= using dralis 

- cutting fluids 


- switching on the drill press 


Note: Through the use of the PLM system the student was ex- 
pected to gain knowledge regarding both the safety precautions 
and safe working practice associated with various machines and 
processes used in EDIND 412, Metals and Materials 1. It was 
not expected that the student would, through the PLM system, 


be able to develop his psychomotor skills. 
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information sheet was devised and printed (see Appendix VII). 
This sheet listed the information required by the PLM author 
which included the following: Module Title, Module Number, 
Instructional Unit Number, Item Mastery and Non-Mastery 
Feedback and Learning Resources. The information sheet also 
showed the space allotted by the PLM program for each of the 
items listed. In this way the size of the test item could be 
rigidly controlled and authored into the PLATO system by some- 
one other than the principal author. 

After the author information sheet was printed a minimum 
of two items, one true/false the other multiple choice, were 
devised for each instructional unit. Other test items were 
subsequently developed for each unit. PLM will allow up to 
sixty (60) items per IU. However for the purpose of this 
study the maximum number of items developed for any one IU 
was seven (7). Each test item was then checked for face 
validity and correctness of information. Having completed 
this, the test items and their non-mastery feedback was 
authored into the PLM system. In order to simplify the author- 
ing procedure and the subsequent item analysis each Module 
Wasioiven a number, i2.e. 1, each Instructional Unit a letter, 
i.e. A, and each Item a Module Number, Instructional Unit 
Letter and a number to denote its position in the item pool, 
Den AL. SALter cinalizing the authoring procedure cach 
Module, Instructional Unit, Item and Learning Resource was 
again checked for accuracy and typing errors. Finally five 


(5) modules were set up to check for viability in the PLM 
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mode and conformity to their stated purpose. The Variable 
Management Strategy was also checked and initial student 
reaction guaged through the use of a pre-pilot study verifica- 
tion of the Modules and an initial data gathering session. 
Once the system had been checked seven (7) other Modules 

(for a total of twelve 12) were set up ready for the pilot 


Studye 


DATA COLLECTION 

Data used in this study were obtained from the students 
of EDIND 412, Metals and Materials, 1A at the University of 
Alberta, during the months of January, 1982 (the pre-pilot 
study verification session) and March, 1982, (the pilot study 
session). The data were collected in two formats via the 
PLATO Learning Management System. Format 1, data collected 
for use by both the EDIND 412 students and the course instruc- 
tor to evaluate student progress through each of the modules 
(see Appendices III and IV) and Format 2, data collected 
for use by the PLM author to evaluate the reliability and 
validity of the test items. 

The data collected in Format 1 took the form of statis- 
tics describing the progress of an individual student in terms 
of the body of knowledge on which he was being tested. The 
data compared one test to another so as to provide a basis 
for judgement of the relative difficulty of each test. The 


factors used for this comparison were as follows: 
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1. time to complete each test 
2. the number of tests taken before mastery 


3. the number of students passing the test on the 
£rrst, second. and last try 


4. the number of tests taken before test mastery 
was achieved by all students 


The data collected in Format 2 took the form of detailed 
item statistics, labelled 'qubasic' in the PLM system. They 
were arranged into eight octal words for each test item, (see 
PEgure: 6, See Data, below) with each word containing a 
specific piece of information. The information labels for 
each of these words are listed in Figure 7, Basic Data Format, 


on the following page. 


Figure 6 


Qubasic Data 


WORD qubasic pimtest question 
1 PECOUS eA Lossa0000000)  **2% 0.000 ivedm05 
2 40 7900600 090000000000 nex*s% 4*** 444 2 O5F 
3 AVTUCeSSOD CO 0OO00000F xx * AAA KEEK SEZ 
4 553 3405:05436504338555 {ath te tke Ane % 505/13 /82 
5 DOCS ASU BO S00) 5044451) Sane eee eR 1. 0529 
6 06040032000000000202 —*** 02 000 matdzbb 
7 00000000000000000002 y) OF O00 
8 00000000000000000000 0 0.000 
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Figure 7 


Basic Data Format 


module name 

IU name 

question name 

date question was presented 

time of day question was presented 
miscellaneous information 

answer given 


second answer word 


(PLATO Aids, The PLATO IST, Control 


Datar COrpOracion, 1.a.) 


Word six listed in the gqubasic data format (above) as 


miscellaneous information contained in a 'packed' format other 


atem ana testuintormnacLlon... This information was. a, Eollows: 


1 


number of tests taken, including current on 
this module by this student 

question type, 1 = multiple choice, 2 = true/ 
false, 3, = matching, 4 = totch, 5 = numerical 
off-line flag, 1 if test taken off-line, else 
0 mastery flag, 1 if mastered, 0 if not 
mastered 

time to answer question in seconds 


question point value (signed) 
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6 points earned on this question (signed) 
7 question number as presented to student 
8 unused 


9 physical course number 


(PLATO Aids, The PLATO IST, 


CONCrOL Data Corporation; n-d.) 


ITEM ANALYSIS 
The methodology devised for item analysis in this study 
was based on that described by Denenberg (1977) and Husec and 
Str orn (1968).. The analysis was divided into five steps 
as follows; 
ie ~thesterans ation. of PLATO data into “ai format 
consistent with that able to be ready by the 
Statistical Package for the Social Science 
(SPSS) analysis program 
2 the development of an SPSS file 
3 the development of an item frequency table 
4 the grouping of students by their relative 
observed score 
and 
5 the development of an item analysis table 
Step b,the translation of sPRATO data fromthe sPLATO 
Cyber 170/720 computer to an Amdahl 470 V.8 computer operating 
under the Michigan Terminal Systems (MTS), was accomplished 
by the use of a Fortran program written by the Instructional 


Systems Group, Computing Services, University of Alberta. 
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This program was designed to place into a number of thirty- 
two bit words, on the Amdahl 470 computer, the following 
information derived from the PLATO data: 
1 student name 
2 Module name 
Sem lnstructional.Uni t.name 
4 Item name 
d= testudate 
6 test time 
7 number of tests taken on that Module 
8 item type 
9 the off-line flag 
10 mastery flag 
11 time taken to answer the item 
12 the item number 
13 the course number 
14 student item response 
Step 2, the development of an SPSS command file (see 
Appendix VIII). This development took place by devising a 
series of two and three way crosstabulation tables. The 
SPSS subprogram crosstabs however has a number of limitations. 
These are: 
"the maximum number of dimensions available 
for any crosstabulation is ten. This means that 
control variables can be stacked eight deep but 
no more than eight control variables may be 
involved in any set of subtables" 
(Nie. IN; Hs, hadina Holl. C.,-Jenkins, 
iC.,. steinbrennem, Ka, -& Bent, D.i., 
SPSS Statistical Package for the Social 


Sciences. New York: McGraw-Hill Book 
Company, 19/55. Pp. 243) 
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Because of these limitations a number of ‘select if' and 
‘*select 1f£' cards were then used for processing only those 
cases for which certain criteria were met, i.e. module number 
and the number of tests taken by all students on that module. 
Nie et al. write, 


"There may be times in dealing with large 
files when the researcher wishes to drawa... 
Sample of cases ...or to select some specified 
Subgroup of the sample for analysis. The .->. 
data selection card ...'select if' provide(s) 
the SPSS system with the capability ...of 
selecting ...cases from the users file. (The) 
card exists in both a permanent form and a 
EempOLrary OLustarreda form ... ('*select.1f').. 
(This) provides the user with the ability to 
create permanent files of ...selected ...cases 
and to temporarily effect those changes for 
use with some particular subprogram within 
the system" 

(IbidepelZay 


Step 3, the development of an item frequency table 
devised after that developed by Denenberg (1977). This table, 
Shown as Table 1, ‘Item Frequency Table, following page, was 
used to present information from all twelve safety modules. 

It listed all items presented to the student and the answer 
frequencies given by them for a first test on any one module. 
The table was also used to list similar information from 


subsequent tests on the same module. 


al Pe doit i 
pon elite patanone 


SO rwee ae ea 


—e olution . 


a Rs 7 a in etka sa. Om 
| pin We "1 ick Bie i apidras 0 we an om 
Beas ipege oma HeLee od. 7o. ‘ errr? 6 
1 eke Bt 08 ‘eye |? ieee ole bo gee 
i Lvov 0 pedi! . i Lveo| Mi leoe fon) ateh 

ne ape i ee ne i Lu chavegws oy tot cia bw wanes San \ +2 » - 2 
‘(ortt) «(of 2F cea Ht me? seuss Jw pabioeew 
me ’ bis io, Soire@ a gerry at ebb ine 6; 59 

rs myen | omg ate Prag: oo rts ee 0. vis oqmed 
as yrhiids. eit new, fey acs aebivesq ta. 
ers ‘. \Brocktvert ete! mi he. beg PS be aaaritidencds 4 eae 
$08: aeneters ‘eahe Goeth iy: \ bia srogen? oF 
nical tw ne +00 sh haa " uLsx9 in ‘ng emat By iw 6 
) Simecre ye . oe 

te ng bith CF , 7 an a 


a 


“alas youn cori mast. ae ae Bweerots ara ote Pa gata 


ota, eee oe remy | puedo ca Bitgoton et: a sala 
" co sme ‘4 griwatia® «shea ‘pdaport matt | +f gigs si 
e-oLnta ates wehew ihe, si tio baeatroh Srna 


Wigs, i ey ih 


zewene, atta cd semis ate’ 
. oe ee 
ws oa ht ! wm a a 


AY 


moet | oat sal tata ie 


olobem oom ne 


67 


Table 1 


Item Frequency Table 


Multiple . 
MOuestion Choice Percent 
Item 


XA 


The question name is shown in the first column with the 
response to that item shown in columns AX through XD 
Columns AX and BX denote a true/false type item with AX repre- 
senting the answer true and BX representing false. Columns 
XA through XD represent the four distractors of a multiple 
choice item. The column frequency lists the number of students 
who attempted that item. The column percentage correct shows 
the percentage of students answering the item correctly. 

Step! 4,) the placing of students into groups by relative 
observed score. }] This table (Table 2, Student Score Table, 
following page) was constructed after that developed by Husek 
and Sirotnik (1968) and like the frequency table, lists all 
items presented to the student and answer frequencies given 
by them for a first test on any one module. The table was 
also used to list Similar information from subsequent tests 
On the same module. 


The item name is shown in the first column with answer 
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Table 2 


Student Score Table 
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frequencies for that item shown in columns A through K, each 
column representing one student. The answer to each item 

is shown as either mastery denoted by the number 1 or non- 
mastery denoted by a zero. The frequency of correct response 
column shows the frequency of response to that item by all 
students. In order to cetermine a student's group the number 
of correct responses to each item by each student must first 
be calculated. This calculation is done by independently 
adding each of the columns A through K with the resultant 
totals being placed in the row frequency correct by student. 
These totals of correct student response are then divided into 
any one of the frequency of correct sta cOcaLS a nlnes 
figure is then placed in the row proportion correct. Once 
this is finalized those students considered to be above 
average and those considered to be below average can be 
caetermined. 

Step 5, the development of an item analysis table to 
contain the information gathered from Tables 1 and 2. This 
table was devised after that suggested by Denenberg (1977). 
Table 3, Item Analysis Table, following page, lists all 
items presented to the student and the answer frequencies 
given by them for a first test on any one module. The table 
was also used to list similar information from subsequent 


tests on the same module. 
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Table 3 


Item Analysis Table 


Frequency| Frequency 
Frequency} Correct/ Correct/ 
Correct/ Incorrect] Incorrect 
Incorrect] Above Below 
All Average Average 
Students Students Students 
R& WS R& Ws 


The question name is shown in the first column, with the 
second column listing the number of times the item was presented 
and attempted by all students. The column frequency correct/ 
incorrect all students, R%, W% shows how all students who 
attempted the item fared on it. R% is a percentage of the 
item frequency column and shows those students who got the 
item correct. W% is the percentage of students who answered 
incorrectly. The next column frequency correct/incorrect 
above average students shows the same information as the previous 
column but only for those students whose score determined by 
the student score table was above the median for the group. 

The frequency correct/incorrect below average students shows 
the same information as the previous column but only for those 


students determined by the student score table to be below the 
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median for the group. The number correct column shows the 
total number of students in the 'correct' category and the 
value on which percentages can be computed to determine the 
number of students who passed the test. The last column, 
number not correct, lists the number of students in the 
‘incorrect' category. These values can also be used to 


determine the number of students who failed the test. 


SUMMARY 

The PLATO system is an interactive computer based educa- 
tional system developed to enable instructors and courseware 
designers to create and manage instruction. 

The structure of the PLATO Learning Management System 
is centered around the Group, A Curriculum, Modules, Instruc- 
tional Units, Question Pools and a set of Learning Resources. 
PLM data collection takes three forms, basic, extended and 
summary and the PLM Variable Management Strategy (VMS) provides 
for the student alternative routes through the curriculum. 

The PLM strategy developed for this study was designed 
around twelve (12) safety test modules with the VMS set to 
randomize all the Instructional Units in the Module selected 
by the student. The criterion level was set at 100 percent 
and the student was required to master all Instructional 
Units before being routed to another module. Each module 
used in the study was developed from two series of safety 
test modules, one a series pilot tested by the Department of 


Industrial and Vocational Education at the University of 
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Alberta, and items written by former EDIND 412 students. 
Data collection for this study was undertaken on two 
levels. The first being statistics describing the progress 
of the student in terms of the body of knowledge on which he 
was being tested. Secondly statistics were gathered on each 
item to facilitate a detailed item analysis. The item analysis 
was divided into five steps as follows: 
Tietne wtratcitation of, (PLATO data, toa, form able 
to be read by an SPSS program 
2 the development of an SPSS file 
3 the development of an item frequency table 
4 the grouping of students by relative observed 
score 
and 


5 the development of an item analysis table. 
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CHAPTER 4 
FINDINGS “OF “THE STUDY 
Tipe lOr -o LUDY 

The pilot study was conducted using eleven (11) EDIND 
412 students in the metalworking laboratory at the University 
of Alberta during the months of January, 1982, (pre-pilot 
study verification session) and March, 1982 (the pilot study 
session). 

The pre-pilot study verification session was set up to 
check the VMS, gauge student reaction and collect data to be 
used in the development of the Fortran translation program, 
the SPSS file, the item frequency table and the item analysis 
table. Five students took part in the session which was 
conducted over the period of one week. 

Prior to the pilot study session a PLATO Instructional 
Systems Terminal was installed in the industrial arts metals 
laboratory. 

One week before the pilot study session was to take place 
all EDIND 412 students were given an explanation of the 
general instructional design, the student path through the 
system and the management strategy employed for each of the 
safety tests. All students at this point indicated a willing- 
ness to participate in the study. Subsequently however one 
student dropped the course and one student made the decision 
not to take part in the study. The students were asked to 
indicate during the study, using the student notes file or 


comments file, any problems they encountered using the system. 
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Finally, the students were asked and encouraged to rewrite, 
uSing the same notes or comments file, any item they deemed 
overly easy, overly difficult or ambiguous. All students had 
prior experience using the PLATO Instructional Systems 
Terminal. 

In order to implement the CMI system without major dis- 
ruption to the ongoing laboratory curriculum, students were 
allowed to use the system during laboratory hours when it was 
most conducive to their own work schedule. At any time a 
student was engaged in a terminal session in the laboratory 
he was observed as to ease of entry onto the system and his 
general progress during each test period. An observation 
notebook was not kept as it was felt the student could use 
the notes and/or comments file to voice his particular con- 
cerns. Also the instructor was present in the laboratory 
during each terminal session held there and therefore able to 
answer any questions that arose, as well as solve any course- 
ware problems that might occur. During the pilot study how- 
ever it was noted that a great deal more time than was origin- 
ally estimated was being taken by the students to complete 
the twelve safety modules (see Discussion page 98 ). Therefore 
students were allowed to use remote sites (away from the 
laboratory) at a time of day best suited to their needs. At 
this point greater emphasis was placed on the need to use the 
notes and comments files. No attempt at observation was made 
during the remote site sessions. 


The remaining sections of this chapter will present the 
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findings of the study, the results of the student question- 


naire and the results of the item analysis. 


FEASTBIGLITY. OF THE SYSTEM 
The pilot study was conceived to explore the feasibility 
of a computer managed instructional system to deliver a series 
of safety tests. Tests that would provide the instructor 
with an indication of student strengths, give the student 
specific remediation when required and provide for that stu- 
dent an inventory of learnings to maintain an awareness of 
personal achievement. Based on this development and imple- 
mentation of the system, the study sought answers to the 
following questions: 
Question 1: 
given the need for instructor feedback 
will computer managed instruction, 
specifically the PLATO Learning 
Management System, provide an 
indication of student strengths, 


difficulties and progress? 


Note: The student records described in Appendicies III and 
IV list eighteen (18) persons on the roster when it has been 
stated (p.66) that only eleven (11) students took part in the 
study. The reason for this discrepancy came about when the 
system was being set up by members of the pilot study planning 
team, who placed their names on a roster in order to test 


the system. At the time of actual pilot study it was found 
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The findings indicate that the PLATO Learning Management 
System will provide feedback to the instructor regarding 
student progress enough to allow a preliminary judgement on 
that student to be made. *"The basis for this judgement is to 
be found in the following student progress statistics: 

- the Student Record Data Display 
(Appendix III, Table 8, page 147) 

- the Course Progress Summary Display 
(Appendix III, Table 9, page 148) 

- Module Progress Summary Display 
(Appendix III, Table 190, page 149) 

and 

- the Individual Sign-On and Lesson Usage Page 
(Appendix III, Table 11, page 152) 

The Student Record Data Display shows the course the 
student is currently working on, if more than one course has 
been assigned, the page also tells the instructor which module 
the student has completed. The Course Progress Summary Page 
lists course mastery or non-mastery and the module progress 
Summary display lists those modules that have been assigned, 
Started and subsequently mastered. These record pages 
coupled with the individual sign-on and lesson usage page 


which lists the days, hours and sessions the student has spent 


that not enough space had been made available on the system 
to allow for a new roster to be set up. Therefore the pilot 
study group (student group) was added to the pilot study 


planning team roster. 
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WOGkING.Onvthe Curricurum can provide for the mnstructor an 
indication of student progress. 

Student strengths and difficulties can be assessed using: 

~ the Course Status Display Page 
(Appendix IV, Table 14, page 157) 
- the Student Module Data (Module B) 
(Appendix IV, Table 15, page 158) 
- Module Report (Module B) 
(Appendix IV, Table 16, page 159) 

The Course Status Display Page of the student gradebook 
lists for the instructor the modules, their availability, 
the student score for each module and the number of tests 
taken for each module. The Student Module Data Pages list 
scores on the first, second and last tests which, when used 
with the Course Status Display Pages and compared to the 
report for each module, provide an indication of student 
Strengths sand’ difficulties. 


Question 2: 
given the need for student feedback 
is the system able to provide an 
inventory of learnings that main- 
tain an awareness of personal 
achievement? 
The results of the study indicated that the PLATO Learning 
Management System did provide enough feedback to the student 


to keep them aware of personal achievement. This feedback was 


achieved through the use of the following: 
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~ the Module Score Display (Mastery) Page 
(Appendix II, Figure 23, page 135) 

- the Module Score Display (Non-Mastery) Page 
(Appendix II, Figure 26, page 138) 

- the Objective Status Display Page (2) 
(Appendix II, Figure 27, page 139) 

- the Course Status Display Page 
(Appendix sul, rigure ll, page £27) 

The Module Score Display Mastery or Non-Mastery Pages 
given at the end of each module provides for the student, 
feedback on the number of modules mastered, this coupled 
with the Objective Status Display Page provides the oppor- 
tunity for the student to view the non-mastered objectives. 
The Course Status Display Page of the student gradebook also 
provides a record for the student of available modules, 
module scores and the total number of tests taken on that 
module. 

The final question posed in this study concerned itself 
with item analysis. Specifically the question asked: 

Question 3: 
given the need for item (question) 
feedback, is the system able, through 
its data gathering capability, to 
provide an indication of overly dif- 
ficult, overly easy or ambiguous items 


in the item pool? 
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Study findings did not indicate that the PLM system was 
able to provide the type of data to allow a viable item 
analysis to be performed using the Statistical Package for 


the Social Sciences (SPSS). 


PLATO QUESTIONNAIRE 

A questionnaire (Appendix VI) was given to all students 
involved in the pilot study. This included the one student, 
who after taking one test on each of the modules declined 
to take further part in the study. The questionnaire was 
divided into four areas of concern to elicit student response 
on the effectiveness, usefulness, Suitability and accuracy 
of the total PLM system. Four other questions of a general 
nature were also added, these questions dealt with both the 
system itself and the PLM safety program. The findings of 


each of these categories are discussed below. 


SYSTEM EFFECTIVENESS (QUESTIONS 1. THROUGH .5) 
Rate the overall effectiveness of the following: 
1 The PLATO system for the safety tests: 


Good Fair Poor 
5 4 3 


2 The safety tests: 


Good Fair Poor 
2 5 5 


3 The ability of the system to keep track of what you 
are doing and how well it is being done: 


Good Fair Poor 
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4 The question non-mastery feedback: 


Good Fair Poor 
9 2 Al 


5 The learning resources: 


Good Fair Poor 
7 S| 2 


To the first question, that of the overall effectiveness 
of the PLATO system to deliver the safety tests, five students 
felt the system was effective, four rated it as fair and 
three felt it was ineffective. Question two dealt with the 
rating of the overall effectiveness of the safety tests, to 
this question two students rated the tests as good, five as 
fair and five as poor. The third question asked the student 
to rate the effectiveness of the systems' ability to keep 
track of what they, the students' were doing and how well 
it was being done (provide feedback). Eleven (11) students 
rated the system as good in this respect and one rated it as 
fair. The fourth question in this series on effectiveness 
asked the student to rate the effectiveness of the non- 
mastery feedback (no mastery feedback was provided throughout 
the tests other than the word 'correct'). Nine students rated 
the feedback as good, two as fair and one as poor. The final 
question elicited response as to the effectiveness of the 
learning resources. Seven students rated them as good, three 


as fair and two as poor. 
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SYSTEM USEFULNESS (QUESTIONS 6 THROUGH 11) 
Rate the overall usefulness of the following: 
6 The gradebook: 


Good Fair Poor 
9 3 0 


7 The learning resources: 


Good Fair Poor 
6 6 0 


8 The non-mastery feedback: 


Good Fair Poor 
3 9 0 


9 The test lock-out: 


Good Fair Poor 
ik 0 al 


10 Do you think the test lock-out time should be changed 
(it is now fifteen minutes) 


Yes No 
i 0 


11 If yes to the above, what should in your opinion 
the test lock-out time be changed to? 


Time in minutes 
PAGO) de Se a TT Wale ie 0, 


The next section of the questionnaire dealt with the 
usefulness of the gradebook, learning resources, non-mastery 
feedback and the test lock-out. Question 6 asked the student 
to rate the usefulness of the gradebook, that is, that section 
of the PLM system that kept track of what the student was 
doing and provided feedback to that effect. Nine students 
rated the gradebook as good or very useful and three rated it 


as fair. Question 7 asked the student to rate the usefulness 
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of the non-mastery feedback. In this respect three students 
rated the feedback as good and nine as fiar. 

The last three questions in this section dealt with the 
test lock-out. The lock-out was placed on the system to 
encourage students to 'master' the test in as few a number of 
tries as possible and secondly to allow students the oppor- 
tunity to review the remediation presented to them after a 
non-mastery attempt. The usefulness of the test lock-out 
was rated as poor by eleven students and good by one. The 
second question concerning the test lock-out asked if the 
lock-out should be changed, to this all students answered 
in the affirmative. The final question in this series asked 
'what should the lock-out time be changed to', nine students 
suggested five minutes and three suggested the lock-out be 


removed completely. 


SYSTEM SUILTABLLITY (QUESTIONS 12 THROUGH 15) 
Rate the suitability of the following: 
12 the test questions: 


Good Fair Poor 
4 6 2 


13 The learning resources: 


Good Fair Poor 
5 6 a 


14 Should other learning resources have been given? 


Yes No 
fl 5 


15 If yes to the above what other learning resources 
should have been used? 
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Test CAI lessons Audio Visual C/R Teaching 
ai A 2 
Example Example Example Example 


Question 12 in this category asked the student to rate 
the suitability of the test questions (items). Four students 
felt that the items were good, six felt they were fair and 
two considered them poor. Question 13, that of rating the 
Suitability of the learning resources, five students thought 
them good, six fair and one poor. The third question in this 
series asked if other learning resources should have been 
given; seven students responded yes and five no. The final 
question in this series asked the student to list other 
resources he felt should have been used in the safety course. 
Three choices were provided for the student to choose from, 
branching to other safety tests, branching to computer 
assisted instruction lessons (CAI) and branching to audio 
visual safety material, a fourth category was added to the 
questionnaire by the students themselves, that of classroom 
teaching. Eight students declined to respond to this 
question. Of the four that did, one suggested branching to 
other safety tests, one to CAI lessons and two to classroom 


teaching. 


SYSTEM ACCURACY (QUESTION 16) 


16 Rate the accuracy of the learning resources, did they 
give adequate and clear direction? 


Good Fair Poor 
5 6 cL: 
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The only question in this category asked the student to 
rate the accuracy of the learning resources on the basis of 
adequate and clear directions. Five students rated them as 


good and six rated them as fair. 


SYSTEM AND PLM PROGRAM (QUESTIONS 17 THROUGH 20) 
17 Did you find the CMI (safety) PLATO terminal work: 


Too much About right Too little 
‘| 5 0 


18 How do you think the PLATO safety program helped 
you learn: 


Excellent Good Fair Poor 
0 7 3 Z 


19 Rate the quality of the PLATO safety program: 


Excellent Good Fair Poor 
0 L. 3 Wy) 


20 Is the PLATO safety program a good way to check for 
safety knowledge? 


Yes No 
6 6 


Why ? 


The first guestion in this series asked the student to 
indicate if they found the CMI (safety) PLATO work too much, 
too little or just about right. Seven students suggested 
it was too much, five thought it about right, none considered 
it too little....The next, question, that of. asking the student 
to rate how the safety program helped them learn, elicited 
the following responses: no student thought the program 


excellent, seven thought it good, three thought it fair and 
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two thought it poor. The last question in this series asked, 
‘is the PLATO safety program a good way to check for safety 
knowledge?'. To this six students answered in the affirmative 
and six said no. A second part to this final question asked 
Simply 'why'. To this responses varied from "the program 
is overly time consuming and very frustrating" to "it (the 
program) ensures that a person knows metal (working) procedures 
and brings out those procedures that may be missed in the 
textbook", At the end of the questionnaire the students were 
given an opportunity to make suggestions for improving the 
system. Some of those suggestions were as follows: 
- rename the modules ‘resource units', remove 
the test lock-out and use the questions as 
a method of presenting safety information 
- remove the 100 percent criteria 
- remove such things as 'personal clothing' 
(safety), eye safety and long hair safety 
from each module and place them into a 
Single module dealing with personal 
safety 
- allow page back to CAI lessons for such 
things as lathe cutting speeds and feeds 
and depth formula 
- have non-mastery students retake only those 


objectives he missed 
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SUMMARY 
Students rated the PLATO system as effective for the 
delivery of the safety tests, the ability of the system to 
keep track of what the student was doing and how well it was 
being done, the non-mastery feedback and the learning re- 
sources. The overall effectiveness of the safety tests was 
however only rated as fair. 

In terms of system usefulness, students considered both 
the gradebook and learning resources as useful. The non- 
mastery feed back was seen as fair and the test lock-out 
system as poor. The suitability of both test items and 
learning resources were considered fair. 

The accuracy of the learning resources were rated as 
fair and a number of students found the safety tests too 
much in terms of actual work. The learning aspect of the 
tests were seen as being good and 50 percent of students 


felt the program was a good way to check safety knowledge. 


PLATO DATA ANALYSIS 

Before commencing with a full and complete analysis of 
the data set, a sample set of data collected from the January 
pre-pilot study verification session were analyzed using the 
Statistical Package for the Social Sciences (SPSS). This 
analysis showed a small number of ‘suspect values', however 
it was felt that a larger and more complete analysis would 
allow these suspect values to be discounted. 


The complete data set derived from the PLM system were 
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collected onto four PLATO data files, ED211DFI, ED211DF2, 
ED211D03 and ED211D04. At the time of full data analysis 
these files were transferred from the PLATO Cyber 170/720 
computer to the Amdahl 470 V7 computing system operating under 
the Michigan Terminal Systems (MTS). Once transferred to the 
Amdahl system the PLATO data, uSing a Fortran translation 
program, were set up into a number of thirty two bit words 
that contained the following information: 


student name 


- module name 

- instructional unit name 

- item name 

- test date 

- test time 

- number of tests taken on that module 

- item type 

- the off-line flag 

- the mastery flag 

- time taken to answer item 

- the item number 

- the course number 

and 

- student item response 

After translation, a small amount of the data set from 

two modules, the Engine Lathe (MO2) and the Pedestal Grinder 
(MO5) were analyzed using the SPSS analysis program. The 


Suspect values found in the sample analysis of data, those 
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expected to be discounted in the full analysis, compounded 
themselves and other suspect values became apparent. These 
suspect values were: 

- multiple entry of all test data for a number 
of modules 

- multiple entry of first test data for some 
students on some modules 

- all student data missing for a number of students 
On a number of modules 

- student data missing for test 1 for a number of 
students on a number of modules 

- discrepancy between the time the item was 
presented to the student and the time taken 
by the student to respond to that item 

- item response shown as incorrect when item 
was answered correctly by the student 

- item response shown as correct when item 
was answered incorrectly by the student 

- multiple correct answers shown when only one 
response was correct 

- an inconsistancy in the randomization of items 
from each instructional unit 

and 

- responses from both true/false and multiple 
choice items attributed to the same item 

These data problems prompted a review by the Instructional 


Systems Group of both the PLATO to Amdahl print program and 
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the Fortran translation program. The assumption was made 
that both the SPSS program and the PLATO data collection 
system were working correctly. After review both the print 
and Fortran programs were considered to be working correctly, 
therefore it was felt a validation procedure should be set 
up to assess the data being collected via the PLATO Learning 


Management System. 


PLATO DATA VALIDATION PROCEDURE 

In order to assess the PLATO data collection system a 
new course, uSing the same twelve (12) safety modules, was 
set up in the PLM system. A new roster consisting of six 
students and a new data collection file was also set up. A 
Simple cross-reference sheet was devised so that respondents 
could enter the item number, the item type and the PLATO 
Instructional Systems Terminal (IST) key struck to respond 
to the item as each test was taken. The cross-reference 
Sheet was then used to compare what the respondent entered 
into the PLM system with what the PLM system recorded on 
its data file. In order to check the item response time a 
micro-computer with printer attached was set up with a time 
loop programmed onto the disk. The program enabled an observer 
to record the time at which the item appeared on the PLATO 
screen and the time the item response was entered into the 
IST by the respondent. Times were recorded for each item 
and using the 'hard' copy obtained from the printer, compared 


£0. the times recorded on. the PLATO data’ file. 
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The validation procedure was carried out using six 
volunteers, three inexperienced PLATO users and three ex- 
perienced PLATO users. Only one module the Pedestal Grinder 
(M05) was chosen for this procedure as: 

- it was considered representative of all 
the modules 

- all suspect values noted previously had 
Shown up at least once during preliminary 
analysis 

and 

- the module was considered short enough not to 
disrupt the volunteers time schedule and long 
enough to collect sufficient data 

During the validation procedure the inexperienced PLATO 
users were allowed to make function key errors as it was 
felt this might simulate that which the students had done 
during the original data collection. The experienced PLATO 
users made deliberate mistakes for the same reason. Of the 
Six 'students', one 'mastered' the module in two tests, 
three 'mastered' the module in three tests. One 'mastered' 
it in four tests and one respondent did not master the module 
after four tests. Again this procedure was done to simulate 
that which might have happened during the original data 
collection. 

Once the respondents had completed the validation pro- 


cedure, the data were collected into PLATO data file EDDFTST 
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in two forms, test summary data and gqubasic data, the data 
file was then printed, ready for checking, using a printer 
attached to the PLATO IST. No attempt was made to transfer 
this file to the Amdahl System or to analyze it using an 
SPSS analysis program. After checking the data file EDDFTST 
a number of suspect values were discovered. (See Discussion 
page 101.) 

The original data, stored in the Amdahl system, that 
from the pilot study, were then destroyed. The same data 
stored in the PLATO system was then translated back to the 
Amdahl System, and an SPSS analysis carried out on the first 
test by all students on all modules. The suspect values 
present in the first data set were again apparent. In addi- 
tion one other suspect value presented itself. This was as 
follows: 

- only true/false type items were analyzed 
for test 1 on one module when both true/false 
and multiple choice items had been responded 
Or 

An SPSS analysis was then carried out on the second test 
by all students on the same modules, the following suspect 
value was detected: 

- only multiple choice items were analyzed for 
test two on one module when both multiple 
choice and true/false items had been responded 


to. 
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It was concluded that in view of the appearance of more 
Suspect data a viable item analysis of the data set would 
not be possible. It was further concluded that system faults 
would have to be isolated, analyzed and corrected prior to 
any PLATO data set being analyzed using an SPSS analysis 
program. 

The decision was made therefore not to present the pilot 
study data but rather present a full and complete description 
of that which was to have taken place using ‘Simulated data' 
to show how item reliability and validity might be assessed. 


(See Appendix I) 
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CHAPTER 5 
CONCLUSIONS AND RECOMMENDATIONS 
PURPOSE, OF THE STUDY 
The purpose of this study was to develop and explore the 
feasibility of a computer managed instructional system to 
deliver a series of safety tests. Tests that would provide 
the instructor with an indication of students' strengths, 
give the student specific remediation when required and pro- 
vide for that student an inventory of learnings to maintain 
an awareness of personal achievement. 
The study sought answers, based on the development and 
implementation of that system to the following questions: 
- given the need for instructor feedback, 
will the computer managed instruction, 
specifically the PLATO Learning Management 
System, provide an indication of student 
strengths, difficulties and progress? 
- given the need for student feedback, is 
the system able to provide an inventory 
of learnings that maintain that awareness 
of personal achievement? 
- given the need for item (question) feedback, 
is the system able, through its data 
gathering capability, to provide an indica- 
tion of overly easy, overly difficult or 


ambiguous items? 
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CONCLUSIONS 

The study determined that the PLATO Learning Management 
System will provide feedback to the instructor regarding a 
student's progress enough to allow a preliminary judgement 
on that student to be made. 

The study concluded that the PLATO Learning Management 
System did provide enough student feedback to keep students 
aware of personal achievement. 

The research did not indicate that the PLATO Learning 
Management System is able to provide the type of data to 
allow a viable item analysis to be performed using the SPSS 
analysis program. 

| The SPSS Analysis Program did not show it was able to 
provide an in-depth analysis of the PLATO data nor did the 
results indicate that overly easy, overly difficult and/or 
ambiguous items could be identified. 

The pilot study as a whole cannot be considered a com- 
plete success as it did not answer all the questions raised. 
The study did however point out a number of potential prob- 
lem areas such as the moving of data from one computer system 
to another and the size of the data set. The study also 
raised the question of the data gathering capabilities of the 
PLATO system and its ability to truly randomize items in each 
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DISCUSSION 
The following section outlines and discusses the various 


results arising from the pilot study. 


THE PLM SYSTEM 

The acquisition of knowledge regarding the PLM system 
was found to be a less than easy task. While publications 
on the subject of CMI are numerous, those relating to PLM 
in particular are both scarce and superficial in nature. 
The PLATO system itself contains large amounts of information 
regarding the capabilities of the system, however this in- 
formation is promotional rather than technical and conse- 
quently has little utility for prospective PLATO authors. 

It would also appear that a credibility gap exists be- 
tween that which the system is promoted as being able to do 
and 7that which 1tcan-actually do, and “that’those expecting 
to use the system should reconsider until further evidence 
is gathered to demonstrate that data collected for item 


analysis is not subject to error or random generation. 


AUTHORING THE PLM SYSTEM 
Authoring of the PLM system proved to be a complex 
procedure. Documentation was found to be lacking and that 
which was available somewhat outdated. Function key sequences 


used during the authoring procedure were found to be very 
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complex structures seemingly unrelated to the task at hand. 


RUNNING THE PLM SYSTEM 
Running the system, that is setting up student rosters, 
creating and destroying files after archiving, collecting 
student statistics and overseeing the system while it was 
in use by the students was found to be the most straight- 
forward task assigned to the author/instructor. However 
the function key sequences were again found to be difficult 


and complex. 


ANALYSIS OF PLATO STUDENT DATA 

Analysis of the PLATO data, that is the reading of the 
statistics listed in the student gradebook as well as the 
‘in-depth' analysis provided by the SPSS Analysis Program 
proved in the former case to be a Simple task but one re- 
quiring a great deal of subjective judgement. The latter 
proved to be a very difficult task and one requiring an in- 
depth knowledge of computing systems. Documentation was 
found to be lacking in the areas of student response record- 
ing, the system of PLATO data collection and data packing. 
(See pages 62-63). The amount of data collected by the 
PLM system is also a Significant factor that should be taken 


into account when considering PLATO data analysis. In order 
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to do an item analysis the PLM system must be set to collect 
qubasic data (see Qubasic data and Basic data format figures 
6 and 7, pages 62-63) for each item answered by all students 
in all tests, which in the case of this pilot study amounted 
to approximately 10,000 pieces of data collected. This data, 
in order for an SPSS analysis to be carried out then had to 
be printed (transferred) to another computing system, in 

this the Amdahl 470 V8, a task that required up to 2,400 
pages of disk space. The data then had to be translated into 
a format able to be read by the SPSS program. Because of the 
Size of the data set analysis it was very expensive. Once 
the data had been translated however the amount of disk 

Space was reduced to approximately 800 pages. The size of 
the data set also presented problems with the SPSS analysis 
program in that the size of the tables produced by the 
analysis would not fit SPSS parameters of a standard format 
and therefore limits had to be placed on the program by 


using 'select if' and '*select if' cards (see pages 65-66). 


SLUVENT Usk OF THES PIMVoYo TEM 
For students the suitability and ease of use of the total 
system met a mixed reaction. All students found the system 
easy to use although some of the procedures they were using 
were more complex than they had been using in various CAI 
programs. Students generally however considered the system 


unsuitable for presentation of safety material. From an 
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instructor point of view however, the system was considered 
to be both suitable and useful. The PLM system tested safety 
knowledge in the cognitive domain but as was indicated by 

the number of tests all students had to take before they 
achieved mastery it appeared the pilot group lacked signifi- 
cantly the safety knowledge it was thought they had acquired 


over the period of their undergraduate degree. 


STUDENT QUESTIONNAIRE 

The student questionnaire given to all students involved 
in the pilot study was divided into four areas of concern to 
allow respondents to air views on the overall effectiveness, 
usefullness, suitability and accuracy of the total PLM system, 
questions 1 through 5 dealt with the effectiveness of the 
system; a total of five students thought the system effective, 
four thought it fair and three thought it ineffective. The 
reason for this negative attitude is not clear, however it 
could stem from the fact that the mastery level (criterion) 
was set by the author at 100 percent, a figure that in retro- 
spect was too high and did not allow for the student to make 
an error or allow for an error in meaSurement due to poor 
items. It could be argued that in the area of safety no room 
exists for error, never the less the students involved in this 
study were affected by the high criterion, taking longer to 
complete a test, using a textbook to check possible item 
answers before entering that answer into the PLATO System 


and exhibiting a great deal of frustration if they got the 
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item incorrect. 

A second factor that should also be considered is the 
randomization by the PLM system of the Instructional Units 
and the selection from the IU pool of just one item for pre- 
sentation to the student. Because of the way this part of 
the PLM system was set up (see page 51) a student failing 
to answer one item correctly was forced to repeat not only 
the missed objective or objectives but a test, again selected 
at random by the PLM system that included all objectives in 
the module. This 'retesting' led, on the students' part, 
to a great deal of frustration especially when only one item 
had been missed. Randomization by the PLM System was an 
option chosen by the author for this study to facilitate 


generalization by the student from a "concrete domain to 
performance in the larger universe of knowledge" (Hively, 
1974, p.8). It would seem however that this randomization 
option should be removed from the PLM system as it checks for 
and finds non-mastery flags at the end of the testing sequence. 
In this way a test would then repeat allowing the student to 
correct only those items he missed. As the system is now 

set up the student is never guaranteed the same test and 
therefore stands a good chance of missing one or more items 
So beginning thercyecre™once more. Siher total level ol irus= 
tration this cycle created, was, it was felt, translated by 
the student into a system wide/instructor ineffectiveness 
rather than being seen as the VMS being set too rigidly. 


Student negativity toward the system as a whole was ap- 
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parenurthroughout the stotal questionnaire. When ‘rating 

system effectiveness only two students saw the system as being 
effective, five saw it as fair and five as poor. These re- 
sults were both surprising and disappointing and again the 
reasons are not clear. The negativity is possibly attributable 
to the students' expectations in terms of the type of safety 
test they were expecting. It is possible they were expecting 
tests similar to those given school age students working in 

an industrial arts laboratory, not a series of items dealing 
With safety procedures they themselves as teachers were ex- 
pected to have Knowledge of and be able to follow when working 


in the university laboratory. 


PLATO DATA VALIDATION PROCEDURE 


As noted, pages 86 through 92, both the print and trans- 
lation, programs were, at the time of the pilot study, reviewed 
and considered to be working correctly. The SPSS program was 
also considered to be working correctly. A validation pro- 
cedure was then set up in order to assess the PLATO data 
collection system. A new student roster was set up and six 
volunteers selected to 'run through' one module. A cross 
reference sheet was devised so that data entered into the PLM 
system could be checked with what was being recorded on the 
data file. In order to check item response time, a time 
record loop was programmed onto a micro-computer disk which 
allowed an observer to record the time an item appeared on 


the PLATO screen and the time a response was entered by the 
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respondent. While this method was not in any way an accurate 
method of validating PLATO response time it did serve as a 
way Of gauging the accuracy of the times entered into the 
PLATO data file. On checking the data file at the conclusion 
of the validation procedure it appeared a number of errors 
were present. The Instructional Systems Group (Computing 
Services) then made adjustments (of an unknown nature) to 
both the print and translation programs and carried out a 
Similar validation procedure on all twelve safety modules. 
When their procedure was concluded no errors were found on 
the PLATO data file. They then carriec out an SPSS analysis 
on the data set, again no errors were detected. (It should be 
noted that at this point in the validation procedure the 
student gradebook and the student records available to the 
instructor were not in question and were not part of any 
validation procedure.) The reasons for this inconsistency 
are not clear. 

As has been described in detail the PLATO data analysis 
was unable to be carried successfully due to a number of sus- 
pect.Velues spresent inthe dataset... The reason_for these 
Suspect values is not clear however four possibilities must 
be considered. . These are: 

1... The PLATO data collection system 
2. The PLATO to Amdahl print program 


STheshOrtran transtacron program 


4, The SPSS analysis package. 
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Only one of these possibilities, the SPSS analysis package, 

least likely to be at fault given its extensive usage in 

a myriad of situations that has caused problem areas to be 

eliminated. Of the remaining three, the PLATO data collection 

system cannot be ruled out until a thorough investigation 

is conducted by systems analysts. It would appear that given 

PLATO's highly technological nature some form of validation 

procedure would have been carried out on the system prior to 

it being placed into a production environment to prevent 

these types of problems arising. Secondly, it would seem 

appropriate for a researcher to consider some form of valida- 

tion procedure for any system Similar to the one described 

in this study if only to eliminate the problem of taking for 

granted that all he reads in regards to that system is true. 
The Fortran translation program and the Amdahl print 

program also present themselves as being a probable cause 

of the data set problems as they were both untried programs, 

the amount of data being transferred from PLATO to Amdahl 

was exceptionally large and the translation program for the 

PLATO data from its packed octal format to a format able to 

be used on the Amdahl system and subsequently manipulated 

by the SPSS program was not validated by the Instructional 


Systems Group. 


ITEM ANALYSIS 
Because of the inherent problems with the data set the 


decision was made to present ‘simulated' data in the same 
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format a full analysis would take (see Appendix I) one module 
MO5 The Pedestal Grinder was chosen. It should be noted that 
the tables constructed for this research reflect that provided 
by the item analysis but are not a true indication of the data 
collected by the PLATO’ system and so cannot be judged as 
displaying true problem items. 

Denenburg (1977) suggests the item frequency table 
(Appendix I, page 118) as being useful to determine answer 
frequencies for a test on one module by all students. The 
table is not designed to provide a definitive answer as to 
the quality (validity and/or reliability) of the item, it 
is designed to highlight only those items that the instructor 
might need to review. 

The student score tables (Appendix I, page 119) devised 
from Husek and Sirotnik (1968) shows module items listed with 
their answer frequencies, from these tables it is determined 
that students C through G and I were considered to be above 
the median for the group and students A,B,H,J, and K were 
below the median for the group. 

The item analysis table (Appendix I, page 121) devised 
fromeapenenburg, (1977). lists, the information, gathered: from -the 
previous three tables. From Table 7 (Appendix I, page 121) 
it can be determined that: 

- item 5A-1 is overly easy as all students are 
getting it correct. This atem should warrant 


some investigation from the author 
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- item 5F-2 is too difficult as all students 
are getting the item incorrect. This item 
should also warrant some investigation from 
the author 

- items 5B-2, 5C-2 and 5D-3 are positive dis- 
Criminating as those students considered to 
be above the median for the group are getting 
the items incorrect. These items could be 
considered to be representative of the total 
best 

- item 5B-1 is a negative discriminating item 
Since those students considered to be above 
the median for the group answered the item in- 
correctly and those considered to be below the 
median answered correctly. This item should 


Warrant some investigation from the author. 


RECOMMENDATIONS 

THE PLM SYSTEM 
That the PLM response recording system be checked, tested 
and documented. 
That the Cyber to Amdahl print program be checked, tested 
and documented. 
That the PLATO data translation program be checked, 
tested and documented. 
That the randomization option in the Variable Management 


Strategy be checked, tested and documented. 
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That PLM documentation be made available, on request, 
LO beth PLM authors and instructors using the PLATO 
system. 

That the study be replicated using a larger number of 
students. 

That the study be replicated so as to judge the educa- 
tional effectiveness of the PLM system. 

That the study be replicated using a micro-computer in 


place of the PLM system. 


THE SAPETY COURSE 
That the pilot study work be continued and expanded to 


twenty-one modules. 


That greater emphasis be placed on the teaching of safety 


and the role of the computer in the acquisition of that 
safety knowledge. 

That the Learning Resources for the safety program be 
expanded to include CAI lessons. 

That the criterion level be set lower than the present 
100: percent. 

That a student lock-out be placed on the modules after 
three unmastered attempts on the same module. 

That the test lock-out be changed from fifteen to five 
minutes or be removed completely. 

That the Variable Management Strategy be changed so 


that only unmastered objectives be retested. 


i r 


36 tothe cope @ ar eo 
<earbs sity» a Pay 7 od ag 8 fetes sit 1 ath 


ok +e a Cyrene sto a ih vote gf intee Leo - vote cae 
oe ; ey eee ee ott, Re ae 


ane _ 7 - J 


% an irene | a seit 
| Wn a - 


a2 bebcmeaee rie beou ian ry r 4 sg wi f Me i okt odd 


} \ oy ee + d 2 { ; i * ed ol i i 
‘outs | to: hitera 2 add no ag ad ein mie Na a 


: a ‘\ i as, - . <egbetwend 9 ‘‘ minke « 
ar 7 nL ae 
ans ee a - WA sm es —. a. vale odd ind 


4 


seseeg odd pha towel oe°% 
- Cy eer pe ty . Pe ae v/. New : 


ny ™, ay ee a = 
nem fe an YY Fi, e. - oF Be 
i : ' 1a ec Tre i 


REFERENCES AND BIBLIOGRAPHY 


Alberta Transportation. Alberta Traffic Accidents 1979 
Statistical Tables, Transporation Safety Branch Research 
and Data Analysis. Edmonton: Queens Printer, 1980. 

RieCeiMan, Dsl. , Alpe lywiieke, & MUrphy, R.r. PLATO rand TLICCLT: 
An Evaluation of CAI in the Community College. Educational 
Technology, “April, 1978, 40-46. 

Allen, M. Computer Managed Instruction. Journal of Research 
and Development in Education. 1980, 14(1), 33-40. 

Aiken, L.R. General Psychology: A Survey. San Francisco, 
Chandler Publishing Company, 1969. 

PACS TS OM, fl tl) ANOS GoOlyn. oc. , LDalgadLd, b.Ray Paden, D.Ws, 
BLIadlLe, Webs, oOLDeL, U.R., & Alessi, o.M,. Am Experimental 
Evaluation of a Computer Based Study Management System. 
BoucatLonal Psyenologist, 1975, lls). l61—190". 

Anderson, T.H., Anderson, R.C., Dalgaard, B.R., Wietecha, 
Ped. SLOdte; Wee Paden, DzW., smock, H.R. ; Alessi, 
S.M., [Orper, J.k., & Klemt, b.0.. Computer Based 
Study Management System. Educational Psychologist, 

EI Ta, ell) ee). 

American Association for Health, Physical Education and 
Recreation. Washington, D.C.: Professional Preparation 
in Safety Evaluation, 1974. 

Armsey, J.W. An Enquiry into the Uses of Instructional 
Technology. New York: The Ford Foundation, 1973. 

Avner, R.A. Cost Effective Applications of Computer-Based 
Education. Educational Technology, April, 1978, 24-26. 

Baker, F.B. Beyond Objectives, Domain Referenced tests for 
Evaluation and Instructional Improvement. Educational 
Technology, June, 1974, 10-16. 

Baker, F.B. Computer Managed Instruction. Englewood Cliffs, 
New Jersey: Educational Technology Publications, 

LOWS. 

Blake, R.P. Industrial Safety. Englewood Cliffs, New Jersey: 

Prentice Hall, inc.., +904. 


106 


Pe ee eg 
- i - 0 j 1D : 7 i : a J 
oO ne ae 
107) >, Gels nan! SLPeoe 
a a jiPaN PF 
; we on a eee 
. ; Heat? i ' 
| ee 
it a re) 7 
/ ; u ; mo i, i ; } P 
. ire a 
f * : 
UW 
5 | ; i a | ne 
7 i 


bd pet mr tas Oraad - ed ‘yet i ie at tae 
Leno bssoube 29m if09 “yd Aepcimenae, oats ae %AD ‘Yo vols 
er Pik ‘ cabs ered Pra 
fogaenon Id ; oe not iter eatin t bapansM tH 
— ipas i OBER «ndteouts ok <a 
‘(qoeton ea mba ve, as s¥ROL: one {stoned ae ti 
Aas ROMsoe Lye) gD | oer —- paide Lidugt tod6i 6 13 ~~ 
Wed! rahe (ase yee opthil 9» toatesbak yeh? Kom 
Ladromt sequel nd, MR 44 aaah . . et ,tadttok, . BW veibbi co 


a. 


aye 


ste7 aye tremageeM yOu: : dame = jie « 80 nottaulisy 

eh WELLER HC) LL 20GE ptaleotevisyed Surette . 

yates te +n te esata cepa : somal valet alas 
_tewetA | Aol | jcnant ye “. hiahe | -s 6188 a4 7 

beans ; “pga pease ie ails t amet ' ont ‘tedrod aaa 

stented toe Saueveo tt iia eto a 2 shomepanem, yi 

1S gh-9e UE sore 


can ot ignores Pee sek om 


ti 


hel wok seg oo a 
noLseseqoty ania Loree De 


a 


2 MOET aoe re AL | Eoygetzvitect vert 
fao dd sraieten Se, eo thi: eh iets 
ait. wp 
oe ia 


7 ETRE kh Naat a bende ieee 
» ms iia Ye; a | 
a oe aves diag ite reer ; 
+3 sunt fee ten. kee ¥ | 
a icviet ae 

kan kamu od itr mop ue 


oe sonia -sateowrs 


107 


Boley, J.W. Guide to Effective Industrial Safety. Houston, 


Texas: Gulf Publishing Company, Book Division, 1977. 


Boyle, J.E. Individualized Intelligence Testing by Computer. 


Unpublished Masters Thesis, University of Alberta, 


Edmonton: Alberta, (1973. 


Bozeman, W.C. Computer Managed Instruction: 
EASE 


AEDS Journal, Spring, 


State of the Art. 
117-137. 


Buss, A. Psychology: Man in Perspective. New York: John Wiley 


and sons, 1973. 


Caldwell, R.M. Improving Learning Strategies with Computer 


Based Education. 
141-143. 


Theory into Practice, 


LOSOn, L192), 


Calliston, W. Problems and Possibilities for Computer-Assisted 


BOS EEUCE LON. 


NASSP Bulletin, May, 


1981, , 24-28. 


Cartwright, G.F. Social Personality, An Attitudinal Diamension 


of Individual Learning with Computer-Assisted Group 


University of 


Instruction. Unpublished Phd, Thesis, 
Alberta, EdmontonesAlberta, 1973. 
Gaetell, 


Set of Concepts in Mehrens, W.A. 


R.B. Validity and Reliability: A Proposed More Basic 


& Ebel, R.B. Principles 


of Educational and Psychological Measurement. Chicago, 


EP eaen@ 15S. 
Clarke, 
Endustry 1969-1978. 


and Health Branch, Ottawa: 
Ltd. All You Wanted to Know About PLATO 


Control Data Canada, 


And Weren't Afraid To Ask. 


Rand McNally and Company, 


196.9. 6 


R.S. Work Injury Experience and Costs in Canadian 


Labour Canada, Occupational Safety 


Queens Printer, October, 1979. 


Edmonton, Alberta: The 


University of Alberta, PLATO Project, Computing Services, 


August, 19 /ic. 
Controdmbata, Corporation. PLATO 
Publication Number 

Control, DatayCorporation, 
Control Data Corporation. PLATO 
Publication Number 


Control Data Corporation, 


Guide, 


Guide, 


Learning Management Instructors 
97406300B. St. Paul, Minnesota: 
Poo cr 

Learning Management Authors 
97406200. 
pe NWA 


St. Paul, Minnesota: 


“techno date ao a wut | 
| ‘Re 008s spo kina 


ip 
heen 


as aéasdeaeinighaty eit ‘eal hthais 
ce rca | 

nowaceqms ke Sanita iten BA wat tdaae 
quot) maenroneulinaitesig A " he 


te le netene 
| ae ve teh 
os Lae nll banndess. rs agp ha 


8 


saatonas 8 Diane a eM ie 


mi way ome *n a 


Bi oe 


108 


Cronbach, Ue.) Leste Valzdation in Thorndike, R.l.e Ed. 
Educational Measurement. Washington, D.C.: American 
Counci on Educatvon, C97 L. 

Gureton, 1.8. The Derinveion and. Estimation.of Test Reliability 
in Mehrens, W.A., and Ebel, R.L. Principles of Educational 
and Phychological Measurement. Chicago, Illinois: Rand 
McNally and Company, 1967. 

Davies, R. The Computer in Performance and Instruction: or 
How To Tell The True Color Of A Chamelon. NSPI Journal, 
November, 1979, 7-12. 

Dean, A.K. Personal correspondence, February, 1981. 

Denenburg, S.A. Alternative Curriculum for Computer Literacy 
Development. Unpublished Doctoral Dissertation, 
University of Massachusetts: Massachusetts, 1977. 

Denenburg, S.A. Alternative Curriculum for Computer Literacy 
Development. AEDS Journal, Winter, 1980, 156-173. 

DeReamer, R. Modern Safety Practices. New York: John Wiley 
and, Sons, “Inc., £958. 

Dougherty, A.J., & O-Nevl, H.F., dr. AneOverview of Training 
and the use of Educational Technology in the Department 
of Defense. Educational Technology, March, 1978, 5-10. 

Doughty, A.E. The Use of Reading Difficulty Indices for 
Predicting Test Item Difficulty. Unpublished Masters 
Thesis, University of Alberta, Edmonton: Alberta, 1982. 

Dyer, C.A. Preparing for Computer Assisted Instruction. 
Englewood Cliffs, New Jersey: Educational Technology 
Publrvcacrons ploy 2. 

Ebel, R.I. Some Limitations of Criterion Referenced Measure- 
Ment in (Bracnuy, GoN., HOPKINS, -h.., 6 Stanley, J C.; 
Ed. Perspectives in Educational and Psychological 
Measurement. Englewood Cliffs, New Jersey: Prentice 
Mattie LC yy ee 

Ellis, A.B. The Use and Misuse of Computers in Education. 

New York: McGraw-Hill Book Company, 1974. 


wworesbt sedlnigihie £08 mis : 
(mokssixebels | ba 
| ever sesopitossiia 
roma. sea hc” sea 
| Street yoner eae 2 
a wtih” ehioe weit wm | 


eine 1 Hoty. aS aide He 
ed bead a cient exe 


+f mtr) ‘estes ee: a 


oe eeeer ati! mine 


after es eRe 


102 


Eisele, J.E. A Case for Computers in Education. Journal of 
Research and Development in Education, 1980, 14(1),1-8. 

Evans, C. The Micro Millennium. New York: Viking Press, 
orion 

Ferguson, G.A. Statistical Analysis in Psychology and 
Education. (4th, Edition) Montreal: McGraw-Hill Book 
Company ,4.19 76. 

PlOrLoveA. Uy eeuovatTera,;—G. |. carety Bducation. (3rd Edition) 
New York: McGraw-Hill Book Company, 1969. 

Frenze, R.J. Guide to Occupational Safety and Health 
Management. Dubuque, Iowa: Kendall/Hunt Publishing 
Company ,v Lovo. 

Furlong, F., & Miller, W. Diagnose: Computer-Based Reporting 
of Criterion-Referenced Test Results. Educational 
Technology, June, (1978, 37-39. 

Government of Alberta. Report of the Industrial Health and 
Safety Commission. (The Gale Commission) Edmonton, 
Alberta: Queens Printer, 1975. 

Gagne, R.M. Essentials of Learning for Instruction. Hinsdale, 
Illinois: The Dryden Press, 1974. 

Gagne wh. Mo os Briggs nu... Principlessot Instructional Design. 
New York: Holt Rinehart and Winston, Inc., 1974. 

Gardner, J.E. Safety Training for the Supervisor. Reading, 
Massachusetts: Addision Wesley Publishing Company, 
L269%. 

Hazen, M. An Argument in Favor of Multimethod Research and 
EValuatronewnecAlL and CMI.“Instrauction.) AEDS Journal, 
Summer, 1980, 275-284. 

Heinrich, H.W. Industrial Accident Prevention. Toronto: 
McGraw-Hill Book Company, Inc., 1959. 

Hicks mbt. oe lnwao. cl ne, TeaCnemsand Lhe. COMmpucer. 
Toronto: W.B. Saunders Company, 1972. 

Hively, W. Domain Referenced Testing: Part One Basic Ideas. 


Educational Technology, June, 1974, 5-10. 


ie 


= afi a 


7 | * "hae  elNeror nh 4 
tod LEdiewe som ‘Line | 


‘cae te) -aolsecuet, ‘yoaka 


LEK ‘idedavercetet to aalqioniy’ , Od,  eepkse a: eis ” 1p6O 


io teon bas Coon ul 4) ot abtud 
pride Ltda ft Saba Saeed sneR, AteM 

La Atel ‘ 
ba i 20gen er) ovaluadoepac oepe i.) ‘seth a ah 
Lenat teoyb oo ivead Tent bene riot ad-aotyetey9, to 
| : ee pT el » a wwolsoet 
Bom, dt taei isbateobat, and, od SyOqeR eatedls to 200 

| eos noa: Gdretmes erag a) Ro ee immer, Wolss 
} ! ree ete ty anasug rertedis 
slabonhe snoitogzteat 2A pa bersiant yo elsidapes® iM. \eape 
| aXe. eRe feat eit? eloailtx a 


_ 


ry ex , ed: ont an. kMt Baw, stehdensa 4 Low stto¥ mi 4 


panhagn congue ota: ‘seh paddies T vista ake i x88 


i Dial eneeehiaes, ” sont 
bee: dasininilh anbaaha tia: 20% or oe 
tei 
Sceahde wht mpatonateHe 


° - 


110 


Husek, T.R., & Sirotnik, K. Item Sampling in Educational 
Research: An Empirical Investigation. Chicago: 

American Educational Research Association, 1968. 
(ERIC Document Reproduction Service Number ED 013 975) 

Instructional Systems Group, Edmonton, Alberta: The University 
of Alberta, Computing Services, Personal correspondence, 
1981-1982. 

Jeldon, D. Sample Safety Test Items, Safety Supplement 
number 4, New York: Bureau of Industrial Arts Education, 
die S(O) 

Kalisch, S.J. Some Directions of Present and Future Computer 
Managed Instruction Systems - Why the Differences? 
Journal of Educational Technology Systems, 1978-79, 
TeS);, 209—2L9o° 

Kansup, W.A. A Comparison of Several Methods of Assessing 
Partial Knowledge in Multiple Choice Tests. Unpublished 
Masters Thesis, University of Alberta, Edmonton: 
Alberta, LOS. 

Kearsley, G.P. A Study of Learner Control in Computer Based 
Pnstruction., Unpublished Phd. Thesis,” University of 
Alberta, Edmonton: Alberta, 1978. 

Kiecekar Wok.) Nite, Nol. ; © Hadlalr Hull, Cy. Statistical 
Package for the Social Sciences Primer. New York: 
McGraw-Hill Book Company, 1975. 

Knapper, C.K. Evaluating Instructional Technology. New York: 
John Wiley and Sons, 1980. 

Krathwohl, D.R. & Payne D.A. Refining and Assessing Educational 
Objectives in Thorndike, R.I. Educational Measurement. 
Wastinocons D.C. 7. oer L Cal | COUnCMINOn MROUCALLON, L9/.). 

Kuder, G.F., & Richardson, M.W. The Theory of the Estimation 
of Test reliability in Mehrens, W.A., & Ebel, R.L. 
Principles of Educational Phychological Measurement. 
Chicago, Illinois: Rand McNally, and Company, 1967. 

Leith, G.O.M. A Handbook of Programmed Learning. University 


of Birmingham: Robert Cunningham and Sons Ltd., 1966. 


“yeve-Eio as tocimumt 3 i 
pihereviad edt sed e . aoe 


ote ewe ape Tia 
Saopasieti td ori? ys tl a dl a 
er ares EE ae ts pet len 


yilenoeen’ 0. ‘ebodeon rene 


heda tidugqat) anne soiodd ” | byl oe 
riiocincae, Saal bs 4 ‘ A 


posse 2 aang ‘at ee 


EEL 


Lord, G.F., & Novic, M.R. Statistical Theories of Mental Test 
Scores. Massachusetts: Addision-Wesley Publishing 
Company, 1968. 

Lien, A.J. Measurement and Evaluation of Learning. Dubuque, 
Iowa: Wm. C. Brown, Company Publsihers, 1980. 

Lippey, G. Computer-Assisted Test Construction. Englewood 
Cliffs, New Jersey: Educational Technology Publications, 
1974. 

Livingstone, S. Criterion Referenced Applications of Classical 
Test Theory. Journal of Educational Measurement, 1972, 
OCT) 2 el 32512 

Lysaught, J.P., & Williams, C.M. A Guide to Programmed 
inisiecuction. Neweyvyork: John Wiley. and Sons, Inc., 1963. 

Mager, R.F. Measuring Instructional Intent. Belmont, 
California: Ferron Pitman Publishers, 1973. 

Magidson, E.M. Student Assessment of PLATO: What Students 
Like and Dislike about CAI. Educational Technology, 
August e978.) Sa. 

Miller, J.I. Using Simulation Techniques to Change Attitudes 
of Students Enrolled in General Safety Education in 
College. Morgantown, West Virginia, 1969. 

Morgan, G.A. A Criterion-Referenced Measurement Model With 
Corrections for Guessing and Carelessness. Victoria: 
The Australian Council for Educational Research Limited, 
IEC MAS) & 

National Safety Council. Accident Prevention Manual for 
Industrial Operations. U.S.A., 1974. 

Nite’ NSH diad iia. Avaya Jenkins meu aca Steinbrenner; Kh, 

& Bent, D.H. SPSS Statistical Package for the Social 
Sciences. New York: McGraw-Hill Book Company, 1975. 

Page, M. Teaching, Technology and Terminal Tutoring. SASTA, 
Apmitien LOS Vae2 hoe, 

PLATO Demonstration and Evaluation, The University of Alberta, 
Edmonton, Alberta: Computing Services, 1978. 


a 7 ee 7 mn - a ; ) 


, 


= Bs 
peel ‘Kedaee te | 
lar 
ws r ee Aa i : ‘4 
pie. ‘ pieeed Xo m i — on 
vs O0eE azedint pines 


Bdowe tons. * ao! zorradaneth 


enoltaotirdua veolonsow : i” 


fs oieaslo ta ‘witaltsatiqgk feritied; , 
sver ytoememedeM he. re “te Sencngot toe? jae 
| i re 

bemmisxpox’ of sie he 3 eee: a wa aT » 
. oar ROR Bre yore ‘aitor. Styx eit no keine ant 
stoomted stoegar thugtzoustent yi bis ea OR oon 
Bayt erode hie MTL se oh Sarena 
sinabode Jaret ‘OTAMT 3a, dopmaesaes dinby FB. U8 38 
wpotonstge? . Lat: Po EAD goods wed het bre, oat 
«Rind £ OF er er ae 
PehuslIIA. Spee i) oo neuptoigat Mls. lunes pated, -t.0 4% 
ae 200% yietes, Lark ach at Bei Loses esanhuse 30 
- hana baie paveneres parr 


,Eaea! 


7 neu " palatiaenloss re ce 
» oa cot Lanbat orsenemt Aen seat) ’ ; 


=a 


2 


PLATO Aids. The PLATO Information Systems Terminal, Control 
Datalcorporation, sn..d. 

Pogue, R.E. The Authoring System: Interface Between Author 
and Computer. Journal of Research and Development in 
BAUCAETON lO COVmLe 5 /—6.8-, 

Popham, W.J. Criterion Referenced Measurement. Englewood 
Cliffs, New Jersey: Prentice-Hall, Inc., 1978. 

Popham, W.J. Criterion Referenced Measurement. Englewood 
Cliffs, New Jersey: Educational Technology Publications, 
Oe 2k, 

Popham, W.J. Evaluation in Education, Current Applications. 
Berkeley, California: MrCutrhan Publishing Corporation, 
IPS VAS Te 

Popham, W.J. & Husek, T.R. Implications of Criterion- 
Referenced Measurement in Bracht, G.H., Hopkins, K.D., 

& Stanley, J.C. Ed., Perspectives in Educational and 
Psychological Measurement. Englewood Cliffs, New Jersey: 
Prentice-Hall winc.,. 19/2. 

Sax, G. Principles of Educational Measurement and Evaluation. 
Belmont, California: Wadsworth Publishing Company, 

En Citgs, Los 

Seidel, R.J., & Stolarow, I. An Heuristic Meta-Model for 
Computer-Managed Instruction. Journal of Research and 
Development in Education, 1980, 14(1), 16-32. 

Shields, M.E. An Item Analysis Method for Evaluating Competency 
Based Test Items. Unpublished Masters Thesis, University 
OfsAlLberta, ecmonton: Alberta, 19.73. 

Splittgerber, F.I. Computer-Based Instruction: A Revolution 
in the Making? Educational Technology, January, 1979, 
30-43. 

Spuck,.D.W., & Bozeman, W.C.. Pilot. Test and Evaluation of a 
System of Computer Managed Instruction. AEDS Journal, 
Palig.19/8, 3ileAL, 

Spuck, D.W., & Bozeman, W.C. A Design for the Evaluation of 
Management Information Systems. AEDS Journal, Fall, 
1980, 30-43. 


‘spc meincien ‘wos es | 
ak jmnney bis 
ims i. HM a ‘Os pote rm 

‘bodwaltgna. dnemeesesett Deora > Te 

oy 1) ECR Bigg sD. Letina tiers rrertet 1 ae 
‘ponwelpet. Jnemesy sem panaaretsd ogee at 
stadia bidet b iclcartl ae cael | emma 
poston ffi aga tae ‘secnenena: ai codsmutand Gat 
Mole th aad pe hd dba sii Neha tig> visto 3 


a wae 
ay) iv e 


ar . er, 

~nod wed be? | a pa ia s dome a «CMa a) 

ude , en Laqolt ‘ge Mil {doa at eens ceaeM hecrero tem - + 

big: peeeyeven ad soy Leqwaentiet eeBR De © viqolass@: ast _ 

syoeset welt — #£5 by wend ETE Atipmery eset, feakploroyed | ; 

STE ort iain trae | 

oo bd se Leg! bas sao yee Kaephatsoste to talqdonts Oye 
ya pals dei tdet | deaarot bie, youn oe 


y Day oy pe ated bait 
o2 Soden 30 sigan ast sworslose a oe 
Germ dounemea a9 Lamivat | ac era aa borpanatheinalaglets 


uss Mae .Aehes youdn’: — at Jasmqoleved) 
gone ese priteinteve wok: bord ett 71 AOnA meacl of HM yal 


wienernem coitgets euatest 


| 


ie a | a ry ae ee e = : - ; - ; 7 : a oh = a eer fT 


— Sues 7 


ni 7 a a " ay 
bee Us ) 


hey un 


ii 


Sraniey, J.C. Reliability in Thorndike, R.f. Educational 
Measurement. Washington, D.C.: American Council on 
Baucawlon, of 97a 

Statistics Canada Catalogue 72-516 (May, 1976). Alberta 
Workers Compensation Board. Claims, Estimated Employees 
and Claim Rate by Industry. Ottawa: Queens Printer, n.d. 

Statistics Canada Catalogue 72-008 (January, 1978) Estimates 
of Employees by Province and Industry. Ottawa: Queens 
Printer. n.d. 

Pea ssGi wn div. AaLOueeis «i, BONN, RoC. & (bales; 0. Rs 
Fundamentals of Safety Education. New York: The 
Macmillan Company, 1973. 

Szabo, M., & Estes, C.A. A Strategy for Developing Cost 
Effective Military Instruction Using Available 
Instructional Materials. (AIM). Toronto: American 
Educational Research Association, March, 1978. 

Taerum, T.V. A Teaching Procedure for Computer Assisted 
Instruction in Mathematics and Statistics. Unpublished 
Phd. Thesis, University of Calgary, Calgary, Alberta, 
OO 

Thompson, B.J. Computers in Reading: A Review of Applications 
and Implications. Educational Technology, August, 1980, 
38-41. 

Thygerson, A.L. Safety Concepts and Instruction. Englewood 
Cliffs, New Jersey: Prentice-Hall, 1976. 

TiCiccoOn,. o.G. Hass loelmprove> Learning. V2. (New York?) RoR. 
Bowker Company, 1971. 

Wesman, A.G. Writing the Test Item in Thorndike, R.L. 
Educational Measurement. Washington, D.C.: 

American Council on Education, 19:71. 

Whitney, A.L. Safety for Greater Adventure in Stack, H.J., 

& Elkow, H.D. Education for Safe Living. Englewood 
Cliffs, New Jersey: Prentice-Hall, 1966. 

Worick, W.W. Safety Education. Man, his Machines, and his 
Environment. Englewood Cliffs, New Jersey: Prentice- 
FS ARS irae 


Beayo, ha ntl be stem: tan ‘ ciate oh 
ERs seen? arsguy | ‘aad t®, siete: vd star | 
senate Bren’ yan eu) godt a: LoleteD BAAMBS 


ti’ “tsaae9 mpi Ds ae 
a, ts we ate, . — 


asian fater oa 


Lg ond aaa 


anoeut) : ‘Bway 10° " ae Satbnt a aun tert wel abefo cont te ro) ree a 
ee TT at nee 
+ ae sue y } on ‘eae ete OTEK ae ’ tes 2u6 at 
oft! etroY wot i ok tno Ta Neo 6 te So" ‘eieyoematinw 
va ie ; eS ever 4 yaa parcial 
Jeod paigelevad sald vedere & .A.3 \aee Pt a .-H 0 
etdetieva pared: SE ggalrd 02 © vies FP evitset ta 
nao ieat veda sot tae A ‘ ale aM ‘fanokdoaraiedt. | 
“a ares yROTEM VO sa jest me ios sacha iatiekteobbe | 
Her eiganh tea an ror eaMbiter9 ror inset x ae nsTOS 
peda tidugdl wks als6se Sena) eehyeus HM ah sotiserdach 
enruedle «eae at ree tan te: iyaier ay tet esa bite. 
a eee | aveb. 2) 
o1san tage We we vor fi - ethan ad hs Aa 3 ange) 4 ta tae quot fl 
,O8GL \Seubean oped on: Sat (sna haeouiet ‘senots oR tgat oo 
Paes | ye | * t¥aet wet 
‘poovntondl , aaksets: rere't bie ‘aoe a an Qe oer 
oa 
| at heat wat ae oot! Lees mieed aeairaapas or. a 
ae ater ‘a o Dae 
em yealknanedta mt —- } wit shal; sane * , 
a de tasshanbte lr: we : | Bou! 
ee ee 
‘yeti ae 7 inane 


he - 


i vio as wt 


Tih, 


114 


Ziel, HoR.' Man Science’ Technology. Edmonton, Alberta: 1.D.B. 


Press, 1971. 


a 
ma sasaedtn 
& yf 


a ra a 
A if rit 


APPENDICES 


LS 


ar An a Me 
ee hy : 

ob vi 
ak Ce) a he 
ra TL 7 

Bit UNE: 

' f 1. 

j a, 


“ 
Py 


=) 


1: 


APPENDIX 1 


ITEM ANALYSIS 


116 


* xrou wacan ou 
antatn Mm 


ITEM ANALYSIS 

A full data analysis was as already noted (page 64) 
not possible. However the following has been constructed 
using 'Simulated data' as an example of the form a full 
analysis would take. One module forms the basis for this 
example, the Pedestal Grinder, Module M05. 

On reviewing Table 4, Item Frequency Table, showing 
Summary results for the first attempt on a test by all 
students, following page, it can be seen that all items pre- 
sented elicited a response from each student, that all items 
5C-1 through 5C-4 were answered correctly by all respondents 
and item 5F-2 was answered incorrectly by all students. 

Table 5, Student Score Table, showing relative observed 
scores for the first attempt on a test by all students, 
page 119, lists relative observed scores for all students in 
test one on the module, Pedestal Grinder. The number one (1) 
represents the student having answered that item correctly 
while the zero (0) represents an incorrect answer. The 
absence of either a one or zero shows that the student did 
not receive that item. From Table 6, Student Score Table, 
Above and Below Average Students, shown on page 120, it can 
be seen that student A answered four (4) items correctly. 
rf this total is’ divided into the*®frequency of correct 
response total for item 5A-1 column 13, Table 5, student A's 
proportion correct score will equal .6 (row 2, Table 6, page 
120). If this same procedure is followed for all students a 
Proportion correct score can be determined for all students. 
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Table 4 
Item Frequency Table, Showing Summary Results 
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Table 5 


Student Score Table, Showing Relative Observed Scores 


for the First Attempt on a Test by All Students 


GOrrece 
Response 
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Table 6 
Student Score Table, Above and Below Average Students 
Student A B c D E F G H AEN petra K 


frequency 
correct by 
student 


proportion 
SOrLeECT 


students 
above average 


students 
below average 


From Table 6 it can be seen that the students' proportion 
correct scores ranged from a low of .6 for student A to a 
high of 1 for students C through G and I. After determining 
the students' proportion correct score the scores were then 
placed into an ‘above' or 'below' average category, rows 3 
and 4, Table 6. The median was .8/7. 

Table 7, Item Analysis Table, summarizing information 
from Tables 4,5 and 6, following page, lists all the relative 
information gathered from those tables. Table 7 indicates 
how overly difficult, overly easy or ambiguous items from the 
Instructional Unit Item Pool may be identified. From the 
Table 1t iS Indicated that: 

- item 5A-1 has been answered correctly by all 


students 
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Table 7 
Item Analysis Table, Summarizing 


intormatLtonerrom lables 4,5 and 6 


Frequency 
Correct/ 


Frequency 
Correct/ 


Frequency 


Correct/ Incorrect! Incorrect 
Incorrect | Above Below 
Question| Item All Average Average 
Frequency] Students Students Students 
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- item 5F-2 has been answered incorrectly by all 
students 

- items 5B-2, 5C-2 and 5D-3 have been answered 
correctly by those students determined in Table 
6 to be above the median for the group and 
answered incorrectly by those determined to be 
below the median 

- item 5B-1 has been answered incorrectly by those 
students determined in Table 6 to be above the 
median for the group and answered correctly by 


those determined to be below the median 


SUMMARY 

The simulated data were set up into three tables to show 
results as if the data were analyzed by an SPSS program. An 
Item Frequency table showing summary results for the first 
attempt on a test by all students. A Student Score table 
showing relative observed scores for the first test for the 
module Pedestal Grinder, a second Student Score table showing 
above and below average students and an Item Analysis table 
listing all relative information gathered from the previous 
two tables. From these tables the response to each item can 
be determined, student proportion correct scores can be cal- 
culated and finally, overly difficult, easy or ambiguous items 


may be identified. 
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THE PLATO SAFETY COURSE 
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THE PLATO SAFETY COURSE 

The following is a description of the PLATO safety 
course used in this pilot study. Shown are the various screen 
displays given to students and the various options open to 
them at each of those displays. Also shown are examples of 
the items used during a test and the various displays shown 
to the student after a test for both mastery and non-mastery. 

When the student signs on to a PLATO Instructional 
Systems Terminal and enters his course name and password, 
he is first presented with a screen display telling him 
his group, the date, the last time he signed on to the system 
and the number of terminal sessions he has had to date. 


(See Figure 8, Student Welcome Page, shown below.) 


Figure 8 


Student Welcome Page 


PLATO Learning Management 


Session * Ii 
Group edind2i1 


Today's date ...... Formal 3, 1932 
Last date on Peril 139, 91962 


Welcome back! 


Press NEXT to continue 
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After pressing the function key 'next' to continue, the 
student is shown the 'menu' or Course Selection Display page 
that lists the modules (tests) that he has ‘tested out' on 
to date. Also indicated, using an arrow, is the module (test) 
that the student should work on next (see Figure 9, Course 


Selection Display, shown below). 


Figure 9 


Course Selection Display (Student View) 


MODULE INOEX 


MODULES YOU CAN WORK ON NOW: TESTS TAKEN 


Drill Press 

Engine Lathe 
Handtools. 

Pedestal Grinder 
Horizontal band saw 
Foundry Furnace 
The Shaper 

Milling Machine 
oxy-acetylene weld. 
The Bandsaw 

The Tool Grinder. 
The Are tWelder 
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i TYPE A-LETTER (to work on that module): > | 


1. To s2e how well you're doing 3. To review instructions 
2. To sign off 4. To read and write notes 
As well as being able to select a test from the Course 
Selection Page the student also has four other options avail- 
able to him. Option 1 Figure 10, Student Progress in Curricu- 


lum, following page, allows the student to see how well he 
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is doing on the course as a whole. The screen display also 
shows the student which course he is presently working on, 
and his status for each module in that course. 

Prgure- 10 


Student Progress in Curriculum (Student View) 


<> test course 


GE Mal al a aC 


© = current course 


Modules: 
O = not started 


QM = started 
B= mastered 


LAB to see the gradebook 
BACK to return to the module list 


Also available is the Course Status Display, Figure ll, 
Shown on the following page. This display allows the student 
to get a much better idea of his progress to date as it 
shows test scores, the number of tests taken, which modules 
are available for testing, the last test taken and the last 
test mastered. 

Once the student has reviewed the Course Status Display 
he can, by entering a module letter, review any one of the 
modules he has been tested on. When the student leaves the 
Course = Status Display ne” 1s-then returned) to the Course 
Selection Display Page (or menu). (See Figure 9, page 125). 

Option 2 on the students' Course Selection Display 
Page allows the student, after reviewing his gradebook, to 
sign off the system. Option 3, 'to review instructions’, 


gives a student an opportunity to review the instructions for 
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Figure 11 


Course Status Display 


/ adind2it April 13, 1982 2:35 PM 
Course 1: test course 
Started: 1718782 Last Test: Module R, 4713782 


Last Mastery: Module R, 4/13782 


Course Score: 8 


MOOUEE’ STeTus: MASTERED SCORE. SI] 
available iS of 138 2 
no testing 67 of 199 1 
~ mastered 4/13/82 1lg80f 19d 1 


available 
available 
available 
available 
available 
available 
available 
available 
available 


HAOVDONTNMOWOADH 


Enter a module letter > 


using the Course Selection Display Page. (Figure 12, Test 
Course Page Instructions, shown following page). 

The Test Course Page Instructions also give the student 
a review on how he should go about choosing a module for 
testing, this is done by pressing function key 1. Option 
2 on the same page allows the student to review the criteria 
for the course and option 3 provides a course introduction 
overview. 

The final option open to the student from the Course 
Selection Display Page is the option to read and/or write 


notes. Notes can be sent to, and received by, other 
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Test Course Page Instructions 


On the MOOULE LIST page, you can: 


e type the module letter to work on that 


module. The “+" marks the module you 
should probably work on next. 


type a number for any of the options listed 
on the page. 


Press BACK now to return to the module list. 
OR 


Press: 1 to get more help, 


2 to see what you have to do to 
master this course, or 


3 to review the introduction to 
this course. 


students or the course instructor. Once the student has 
selected a module from the Course Selection Display Page he 
is presented with a screen display that describes the Module 
EGriteria for that imoaule. (Figure 13, Module Criteria Page, 
shown following page). When presented with the Module 
Criteria Page, the student again has three options open to 
him: he can see all the objectives for that module, take the 
test immediately or choose another module. If the student 
chooses to see the objectives, he is presented with the Module 
Objective Page shown in Figure 14, following page. 

After the Module Objective Page has been presented to 
the student he can press the function key 'next' which will 


then display each objective to be mastered. (Figure 15, 
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Eigure, 13 


Module Criteria Page 


Drill Press 


NOTE! 198% must be achieved on this test in order to 
pass. If you attain the 199% you will be given a'learners 
permit’. If you score less than 190% you will be given a 
list of references. After studying them you may attemp 


the test again. 


WHAT DO YOU WANT TO DO NOW? 
1. See the objectives 2. Take the test 
3. Choose another module 


Figure 14 


Module Objective Page 


Module Objectives Module A: "Drill Press” 


i 
Ze 
3. 
ae 
5. 
6. 
7. 
8. 
9. 
g. 


J 


The most important thing for you to do in this course is 
to master the objectives. 


Press NEXT to see the objectives. 
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Objective Status Display (1) shown below. 


PrgureL5 


Objective Status Display (1) 


Module Objectives Module A: 


“Drill Press“ 


Speed selection for drills. 


Enter an objective number > BACK for more options 
NEXT only for objective # 2 HELP for instructions 


When the student elects to take a test, option 2 on the Module 
Criteria Page (Figure 13, page 129) he is presented with the 
Instructional Unit Testing Strategies Option Display, 

Figure 16, following page. After pressing the function key 
text’ a Begin Test Display Page 1s shown, (Figure 17, fol= 
lowing page) at this point the student has the option to go 
back to the Course Selection Display Page and make another 
choice, once the test is started this option is no’ longer avail- 
able. 


On pressing the function key ‘next’ to begin a test the 
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Figure 16 


Instructional Unit Testing Strategies Option Display 


Testing Procedure for Module A 


This test covers 12 cbjectives. 


To master this module, you must master 12 cbjectives. 


Figure 17 


Begin Test Display 


Your test begins with the next screen display. 


Press ANS to begin this test. 


Press BACK if you don't want to take the test now. 
student causes the PLM system to randomize all the instruc- 
tional units in that module. Once randomized one instruct- 
tional unit is presented to the student, Figure 18, 
Instructional Units Cbjectives Option, following page. From 


this display the function key 'next' is again pressed, this 
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Figure 18 


Instructional Units Objectives Option 


Objective “1C” Module A 


Eye Protection. 


time to cause one item from that instructional unit item pool 
to be presented to the student. (Figure 19, Test Item (1), 
Shown below). 

Figure: 19 


Test Item (1) 


Module A Objective: 1C Question: 1Ci 


Safety glasses need only be worn when using a reamer 
on the drill press. 


TRUE or FALSE 


Press “t" or “f" and then press ANS to score. 
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In order to answer the item presented, the student selects 
the best possible answer choice available by pushing the 
appropriate number or letter key on the IST keyboard. Once 
a choice has been made a check mark appears next to that 
choice. The student can at this point change his answer 

by pressing another number or letter key. Once this selec- 
tion process is completed the function key ‘ans' is pressed 
and the answer recorded in the PLM system file. If the 
answer given was correct the word 'correct' appears at the 
bottom of the screen display (see Figure 20, Correct/Incorrect 
Feedback Option, shown below). 


Figure 20 


Correct/Incorrect Feedback Option 


Safety glasses need only be worn when using a reamer 
on the drill press. 


TRUE or vw FALSE 
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If the item was answered incorrectly the feedback 'incorrect' 
appears at the bottom of the screen display along with other 
non-mastery feedback. (See Figure 21, Question Feedback 
Messages Option, shown below). 

Figure 21 


Question Feedback Messages Option 


Module A Objective: IL Question: tbl1 


The most critical time of the drilling operation 
eccurs when the drill starts to break through - when 
this happens you should 
1) exert more pressure 
2) continue with the same pressure 
3) ease up on the pressure 
y 4) chance drills 


5) stop the machine 


NEXT to continue 


Always ease up on the drill as it starts to break through. 
Otherwise it could jam in the work causing damage to the 
‘drill, the work, or injure the operator. 


Once all the objectives and items in the module have 
been presented to the student he is then given a Module 
Mastery Feedback Display, Figure 22, following page, or a 


Module Score Display Non-Mastery Page (see page 138). 


After the student has read the Module Mastery Feedback 
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Figure 22 


Module Mastery Feedback Display 


WELL DONE! You have achizved 183% on this test and are 
ther2fore eliigible for your ‘learners permit’. Your 
record has been marked accordingly. Please go on to the 
next module. 


Display Page and presses the function key 'next' he is shown 


a Module Score Display Mastery Page, Figure 23, shown below. 


Figure 23 


Module Score Display (Mastery) Page 


Test Results for Module: The Tool Grinder. 


There were 11 objectives tested; 
you mastered 11. 


Test 


Again the function Key ‘next* 18 pressed and the student is 
shown the Exempt Module Page (Figure 24, following page). 
The Exempt Module Page gives the student the option of 
seeing the objectives again, retaking the tests to get a 
better score or choosing another module. Option 3 'see your 
study assignment', Figure 25, Optional Assignment Display, 
page 137, gives the student an opportunity to review the 


learning resource list. (This option was not available to 
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Figure 24 


Exempt Module Page 


The Tool Grinder. 


Since you have mastered this module, you may 
retake the test on a “no-risk” basis. Only a better 
score than previously earned will be recorded. 


All objectives in this module will be tested. 


WHAT DO YOU WANT TO DO NOW? 


1. See the objectives 3. See your study assignment 
2. Take the test 


4. Choose another module 


the students during this pilot study.) If the student does 

not achieve mastery a Module Score Display Non-Mastery 

Page is presented on which his score is displayed. (Figure 

26, page 138). From this page the student has the option 

of first seeing the objectives he missed (Figure 27, Objectives 
Status Display —(2)5) page 139) orsgoing directly to his study 
assignment. (Figure 28, Study Assignment Display Page, 

page 140). The Study Assignment Display Page lists in its 

left hand column the type of material the assignment contains, 
the center column shows the objectives missed and the right 


hand column the number of the objective as it appears in the 
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Figure 25 


Optional Assignment Dispizy 


Your record sous you rest ro 2z8e2igme is 
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Figure 26 


Module Score Display Non-Mastery Page 


Test Results for Module: Drill Press 


There were 12 cbhjectives tested; 
you mastered 8. 


Test 
Mastery 
rT 
g 1 2 3 “ S 6 7 3 SUB e st tal 2 
t 
Your 
Score 


I'm sorry but you have not achieved the 188% pass mark 


required for this test, you should copy down the 
references that will be presented to you in a moment, 


read them, and try the test again. 


NEXT to see which objectives you 
mastered and did not master 


LAB to see your study assignment 
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PEgure 2] 


Objective Status Display (2) 


Objective Status Module A: “Drill Press" 


Y = mastered 
D = not mastered 


- = unattempted 
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The most important thing for you to do in this course is 
to master the objectives. The above table shows you 
which objectives you have and have not mastered as a 
result of the test vou just took. 


Prees NEXT to review the objectives and see your 
mstructor's comments, or 


LAB to see your study assignment. 
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Figure 28 


Study Assignment Display Page 
Study Assignment for Orill Press 


removing sleeves, sockets, drill chucks. 


Clamping Work. 


Using Drills. 
Cutting Fluid. 


Enter an assignment number for instructions > | 


SHIFT-NEXT to list all study materials 


BACK to take a test or review objectives HELP available 
CMI program. The student must now enter the objective number 
of his choice and note down the assignment given. (See 
Figure 29, Learning Resource Assignment, following page). 
This procedure continues until all assignments have been read. 
When the task is complete the student is shown his final 
display page, the Locked Module Disotey, see Figure 30, page 
142. This final page, reviews for the student the module 
criteria and tells him he is locked out of the module for a 
period of fifteen minutes. This lock-out is placed on the 


system to allow students time to complete the given assign- 
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Figure 29 


Learning Resource Assignment 


Study Assignment for Drill Press 


ee removing sleeves, sockets,drill chucks. TEXT 


How to Hold Drills in the Drill Press. 
Walker 34-7 


Fitting Taper Shank Drills 
Repp and McCarthy p.159 


You should complete this activity before 
taking another test on this module. 


NEXT for the next item in this assignment 
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Figure 30 


Locked Module Display 


Drill Press 


NOTE! 198% must be achieved on this test in order to 
pass. If you attain the 198% you will be given a'learners 
permit’. If you score less than 198% you will be givena 
list of references. After studying them you may attempt 
the test avgain. 


You are locked out of this module. You have been 
locked out until 2:43 PM MST 4 4713782 


Because you have taken 1 test on this module in 
which you have not demonstrated mastery, you cannot 
take any more tests at this time. 


You can resume testing after 2:43 PM MST on 47137382. 


WHAT DO YOU WANT TO DO NOW? 


1. See the objectives 2. See your study assignment 


3. Choose another module 
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ments. The student can at this point review the module 
objectives, see his study assignments again or choose another 


module. 
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PLM DATA: INSTRUCTOR ACCESS 
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PLM DATA: INSTRUCTOR ACCESS 

THEVPIMedatatce lection for this ‘prlot study used both 
basic and summary data that was set up to collect data from 
eveny--ces tthe fstudéentetook@up, to .andGincluding *thesfirst 
time the student achieved mastery. 

The raw data is first collected into a data file 
attached to the PLATO group, then when needed by the in- 
structor, collected into a second data file where it is 
sopted into meaningfubistatistics’..\' Data collection is 
carried out by the instructor accessing the student per- 
formance report selection display (Figure 31, shown below) 
and pressing the function key 'shift next'. 

Figure 31 


Student Performance Report Selection Display 


Student performance report selection display 


PLATO: Student. Per formance Reports 


For Student Group: edind211 


In Curriculum: ed2iticur 


Enter report number: 


t. Name Lists 
2. Averaged Progress Data* 


3. Scheduling Reports 
* Summary statistics last collected 4/13/82 12:11 PM 


SHIFT-NEXT to sollect new statistics 
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Once new statistics have been collected, the instructor 
then accesses, via option 1 on the Student Performance 
Report Selection Display, the Student Name List Selection 
Display Page, Figure 32, page 147, from which he has four 
data display options. Option 1 shows the instructor the 
student Record Data’ Display Page, Table 8, page 147. This 
page allows the instructor to review the module the student 
is working on and the course the student is working in. 
Also given is the status of the student regarding the module 
he is working on. The final piece of information given 
on this page is student curriculum mastery, this is indi- 
cated by the check mark on the left side of the screen dis- 
play. From this display an individual student record is 
available. Under option 2 from the same selection display 
page the Course Summary Display Page can be accessed, (see 
Table 9, page 148) this display shows the instructor which 
courses the student has mastered or started, if the student 
is responsible for more than one course. Again an in- 
dividual student record is available. 

Option 3 accesses the Module Progress Summary Display 
Page given with an individual records option. (Table 10, 
page 149). This page allows the instructor to review the 
status of each individual student for each module in each 
particular course the student is involved in. Under option 
3, access is also provided to the Course/Module Reports 
Selection Display Page (Figure 33, page 150). From this 


page all data from a particular course can be viewed, this 
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Figure 32 


Student Name List Selection Display Page 


Student Name Lists 


What would you like to see? 


1. Student record data (class, VMS, 


* 
2. Course progress surmary 


3. Module progress 


x 
Surmary 


4. Last module mastery dates* 


* Summary statistics last collected 


Table 8 


4/13782 


etc.) 


12:11 PM 


Student Record Data Display Page 


Student 
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Table 9 
Course Progress Summary Display Page 


Curriculum/Group: ed21icur/vedind211 5/85/82 12:36 PM 


COURSES: 1g 12 14 
Student Class PANS RANE BR Lead oye ip Bo Proms Lions on} (| hah 4 
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Table 10 


Module Progress Summary Display Page 


Curriculum/Group: ed2itcur/edind2i1 5/85/82 12:36 PM 


* 
COURSE #2 EFGH IJK. 
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module started = mastered 


assigned 


Y = course mastered 


sxx End of Students *** LAB for listing options 
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Pigure, 33 


Course/Module Reports Selection Display Page 


Curriculum/Group: 4d211curvedind211 4/13/82 12:11 PM 


Course #2: “test course” 


ENTER A MODULE LETTER for detailed information on that module 
or enter course level report number (see list below): 


COURSE LEVEL REPORTS 
> 1. Module progress summary 
2. Average module scores 
3. Average module durations 
4. Average module confidence 


5S. Choose new course number 
Now in GRAPHIC display mode. LAB to switch to TABULAR. 


SHIFT-NEXT for next course 
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includes a module progress summary, (option 1) average 
module scores, (option 2) average module durations, (option 
3) and average module confidence, (option 4). (An option 
not in use during this pilot study.) Option 5, choose new 
course number, provides instructor access to different 

PLM courses. The final option, 4. from. the) Selection);Display 
Page, lists the last modules mastered by the student and the 
date on which that mastery took place. 

The instructor also has access as a general guide to 
student progress, a record of lesson usage showing the 
number of times the student has signed-on to the system, 
the time spent during those sessions, the total number of 
IST sessions and the amount of Central Processing Unit 
(CPU) time used. (See Table 11, Individual Sign-On and 
Lesson Usage Display, page 152, From the individual record, 
a group record is also available that shows average times, 
sessions and CPU time used for all students. (See Table 
12, Group Sign-On and Lesson Usage Display, page 152). 

Option 2 from the Student Performance Report Selection 
Display (Figure 31, page 145) gives the instructor a Course 
Program Summary which can be shown in either a graphic 
format (Figure 34, Course Progress Summary Page (Graphic), 
page 153) or a tabular format (Table 13, page 154). Finally, 
option 3 from the same display page (Figure 31, page 145) 
accesses the scheduling reports which allow the instructor 
to check student progress through a curriculum based on a 
starting date, module durations, study times and a master 


calendar of dates. 


wali 290) vapakoous | ae 
okay! ‘nal aia ot 


ier ee opal Wa 
wW eg” SEQ. ah Age 


rT a 


trees uh , on ae van | rete at Pe 


ida 
7 | | _ 


oe, ats : sei ; rz yYi62 a > 1 ree : nokie | 


‘2 wee, Lee #20 iat Mme. eals sesoursank 6 


Py iy quinn bwoda ba we oe ta 4 ao “mo os Loc. ‘ yaas 32038 am 
vt ’ vy ri , j : ‘i a , 
eae oe ae, r oe ae an ¢ 74 og emia to i 4 ini 


6100 arn pees rn Ww te ei ee thos ert bas eacieese TAX 

nee pie ll nif sus wit | a » hese tT -on8) fee ems {oe ! 
SOX eutiev shire oma 1 x: pee fi “pe qs ‘eee eosay . 
tenrs soetees avodte aiid whit be oole al bsooen § 


ee wee) nebo tn the aa, bony eva} pd ons sce 
. 7 


+ 18bs nisin, hantles sah epuan: eoeeel bas sO=mpit QoL, oft 
solsedLan 4 a Po tebe snobuee edt, mow © botage ee 
sluachs sogasiparomi aids Pegs (ear ened me wsirpeny = act 

.kafigee a aes is rk nwosie od nao dois Cones 

7 » picts) oe ame anoryoas se30W02 «ot, 
" tite. . hee ey vet ol coy aah rstudes 0.40: | 


bene vee ie ‘euryt?} oped | 


Table 11 


Individual Sign-On and Lesson Usage Display 


Record usage for group edind21! as of 85/95/82: 
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Table 12 
Group Sign-On and Lesson Usage Display 


Usage averages for group edind2iit as of 84/13/82: 


* Days Hours . Sessions CPU 
Students 18 4 3-98 3 3.6 
Multiples 
Authors 


Instructors 


These statistics include only those records which 
have signed on at least once. 
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Figure 34 


Course Progress Summary Page (Graphic) 


471 o762 12211 PM 
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Table 13 


Course Progress Summary Page (Tabular) 


Curriculum/Group: 2d21bcur-2dind2it Beit Ce Ol 


4/13/82 


Course Progress Summary 


Number of Students (%) Average 
Course | Assigned Started Mastered Score 


138 (19%) 63 


Enter a course number for more information > 


LAB to switch to GRAPHIC display mode 
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eres, 


PLM STUDENT DATA 

The following data are used by the course instructor 
to view and judge student progress. These statistics are 
accessed through the name list option shown in Figure 31, 
Student Performance Report Selection Display, page 145, 
Appendix III. Shown is one student's test results for one 
module assigned during the pilot study test period. Table 
14, Course Status Display Page, following page, shows the 
course status display from the student's gradebook. This 
page lists for both the student and the instructor the module 
the student is working on, or just mastered, the student's 
score and the number of tests he has taken before mastery. 
Table 15, Student Module Data, page 158, shows an individual 
student record for one module giving the date the module 
was last ‘worked on', the time of day the module was accessed 
and the duration in minutes of the first, second and last 
tests taken, the number of tests on that module and when 
the module was mastered. Also listed is the score the 
student achieved on the first, second and last test taken, 
the possible score (criterion) and the percentage score. 
Table 16, Module report, (Module B), page 159, shows the 
group averages for all students over all first, second and 
last tests for that module. Also shown is the number of 
students starting the module, the average number of tests 
for that module and the average number of tests taken before 


mastery. 
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Table 14 


Course Status Display Page 


/ edind2it April 13, 1982 1:27 PM 
Course 2: test course 
Started: 1/22/82 Last Test: Module R, 3785782 
Mastered: 3/7985/82 Last Mastery: Module R, 3/85/82 


Course Score: 
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Table 15 
Student Module Data, (Module B) 


4 edind2ii April 14, 1982 1:26 PM 


Course #2 Module B: “Engine Lathe” 


3 tests taken Module mastered on: 3/93/82 


DATE 1722782 1722782 3/83/7382 


TIME 9:50 AM 18:55 AN 12:48 PM 
DURATION 12 min. 13 min. 8 min. 


EDIT to chanze this record 
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LAB for assignments SHIFT-LAB for group averages 
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Table 16 


Module Report, (Module B) 


Curriculum/Group: ed21icur/edind211 S71376c he tie 
Course #2 Module B 
Students started: 13 Average number of tests: 4 


Average number of tests taken before mastery: 3.989 


Second Later 


Tests Mastered* 


Percent Score 


Average Test 
Duration (min.) 


“Only the first mastery by each student is counted. 
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PLM MODULE DATA 
The following basic and summary statistics are those 
collected by the PLM system during the pilot study session, 
March, 1982. They serve as data on which a judgement re- 
garding the module as a whole can be based, Figure 35, Module 
Progress Summary Page (Graphic), shown below, shows the 


module progress for the safety course number two. 


EYoure 30 


Module Progress Summary Page (Graphic) 


Curriculum/Group: ed21lcur/edind211 4/13/82 12:11 PM 
Module Progress Summary for Course *2 


a = started | =z mastered 


NEXT for next course 
LAB to switch to TABULAR display mode 


Figure 35, above, shows the number of students starting 
and the number of students mastering the module, i.e. for 


module A, 88 percent of the total student group started the 
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module and of that group 77 percent mastered the module. The 
same figure is also presented in a tabular format, Table 17, 
below, and shows in column 1 the assigned modules and in 
column 2 the number of students to whom the module was assigned. 
Under the heading 'started', column 3 the number of students 
who started the module is listed with the number in paren- 
thesis being the percentage of same. Column 4 lists the 
number of students who mastered the module and the relevant 
percentage. The final column, 5 lists the average number of 
tests taken by all students for that module. 

Table 17 


Module Progress Summary Page. (Tabular) 


Curriculum/Group: ed2ilcur/edind2il 4713762. “t23 11 PM 
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Figure 36, Average Module Score Page (Graphic), shown 
below, shows the average module scores for all students over 
all modules for one test. Table 18, following page, shows 
tne vsamesinformatvoueuuma tabular format...) Column i) lists 
the assigned modules, column 2 (right hand side) the number 
of first tests completed and on the left hand side the 
average score for those tests. In column 3, the number of 
second tests is shown on the right hand side with the average 
mark for these tests shown on the left. Column 4 lists, 

Figure 36 


Average Module Score Page (Graphic) 
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Table 18 


Average Module Score Page (Tabular) 


Curriculum/Group: ed21icur/edind211 4713782 12:11 PM 


TEST (avg/n) Average 
Module First Second Last Overall 


Average Module 
Scores (%*) 


Course *2 
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on the right the number of tests taken beyond two before 
mastery was achieved by all students, with the mastery score 
of 100 percent shown on the left. The final column, 5 shows, 
on the right, the total number of tests given for that module 
before mastery was achieved by all students. The average 
score for these tests is given on the left. 

The final graph, Average Module Durations Page (Graphic) 
following page, lists the time in minutes taken to complete 
each module. Table 19, Average Module Durations Page 


(Tabular), page 166, lists in column 1 the assigned modules. 
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Column 2, left side, lists the average time taken for that 
module and the number of first tests completed. In column 3 
the average time is again given on the left hand side with 
the number of second tests shown on the right. Column 4 
lists, on the left, the average time taken over all tests 
after the second test, with the total number of tests taken 


before mastery was achieved by all students, shown on the 
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Average Module Durations Page (Graphic) 
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right. The final column lists the average time taken by 
the student for all tests for that module, with the total 


number of tests given for that module shown on the right. 
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Average Module Durations Page (Tabular) 


Curriculum/Group: ed21licur/edind2i! 4713/82 12:11 Pf 
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APPENDIX VI 


PLATO STUDENT QUESTIONNAIRE 


LG:7 
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PLATO SAFETY MODULES 


Term II 1981/82 


Use questionnaire 


Rate the overall effectiveness of the following: 


1. The PLATO system for the safety tests: 


Good 


ial 


2. The safety tests: 


Good 


Ga 


3. The ability of the 


Fair Poor 


(Coa asad 


Fair | Poor 


Ee) ff) 


system to keep track of what you are 


doing and how well it is being done: 


Good 


ca 


Fair Poor 


C4 ES) 


4. The question non-mastery feedback: 


Good 


keisd 


Fair Poor 


[a] [oe 


5. The learning resources: 


Good 


[ 


Fair Poor 


[ea] aa 


Rate the overall usefulness of the following: 


6. The grade book: 


Good 


EI 


Fair Poor 


a [s] 


7. The learning resources: 


Good 


leat 


Fair Poor 
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8. The non-mastery feedback: 


Good Fair Poor 


= [=] [=] 


9. The test lock-out: 


Good Fair Poor 


= =) = 


10. Do you think the test lock-out time should be changed 
(it is now 15 minutes) 


Yes No 


C) ce 


ll. If yes to the above, what in your opinion should the 
test lock-out time be changed to? 


Time in minutes 


20m 10. 7 55 80 


Rate the suitability of the following: 
12. The test questions: 


Good Fair Poor 


[_] [a] LJ 


13. The learning resources 


Good Fair Poor 


[3 fas} ] 


14. Should other learning resources have been given? 


Yes No 


Ea Ea 


15. If yes to the above what other learning resources should 
have been used? 


Test CAI Lessons Audio visual 
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Example Example Example 
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18. 


19. 


20. 
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Rate the accuracy of the learning resources, did they 
give adequate and clear directions: 


Good Fair Poor 
Did you find the CMI (safety) PLATO terminal work: 
Too much About right Too little 
How do you think the PLATO safety program helped you learn: 
Excellent Good Fair Poor 
Rate quality of the PLATO safety program: 
Excellent Good Fair Poor 


Is the PLATO safety program a good way to check for safety 
knowledge? 


Yes No 


aay fay 


Why ? 


What suggestions do you have for improving the system 
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APPENDIX VII 


AUTHOR INFORMATION AND QUESTION SHEET 
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Note: The Author Information and Question Sheet shown 
on the following page has been Sub 1ect, tO Le= 
duction for printing purposes. This repreduction 
therefore does not reflect the screen space allotted 
by the P.L.M.. program. for each..of the items listed. 


(See page 60.) 
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